We use essential cookies

Please Accept our Privacy Policy

MLOps Engineer

Woven Solutions LLC

Reston, VA • $150,000 to $180,000 / yr • 11/18/2026

Job Description

Job Description
Description:

Role summary:

We’re looking for a MLOps Engineer to own the infrastructure that powers our AI/ML platform. Our products rely on generative AI models - image generation, video synthesis, LoRA fine-tuning, and LLM orchestration - running across GPU clusters, distributed training pipelines, and multi-environment deployments. We need someone who can make all of that faster, more reliable, and easier to scale.


You’ll be responsible for the full ML infrastructure lifecycle: building and maintaining training pipelines, managing GPU compute clusters, automating model deployment, optimizing inference performance, and ensuring our systems run reliably across cloud, and air-gapped environments. You’ll work closely with our ML engineers and researchers to translate their work into production-grade systems.


This is a high-autonomy role on a small team where infrastructure decisions have immediate, visible impact on the product. If you want to build the backbone of an AI company - not just maintain someone else’s - this is the opportunity.


What you will do:

• Design, build, and maintain GPU inference infrastructure on AWS and Azure - including EC2 accelerated computing instances, and SageMaker inference endpoints

• Build and optimize CI/CD pipelines for ML models, from training through evaluation, packaging, and deployment across multiple environments (cloud, air-gapped)

• Automate model deployment workflows - packaging trained models, building optimized inference containers, managing model versioning, and rolling deployments with zero downtime

• Build and maintain custom AMIs and Docker images for GPU workloads using Packer, ensuring consistent environments across development, staging, and production

• Optimize inference performance - GPU memory management, batching strategies and latency reduction for real-time generation endpoints

• Implement monitoring, alerting, and observability for ML systems - GPU utilization, queue depths, model latency, training job health, and cost tracking

• Manage infrastructure as code using Terraform/OpenTofu

• Use AI coding harnesses such as Codex, Claude Code, Pi or Opencode to augment your strong judgement

• Collaborate with ML engineers to debug training failures, optimize data pipelines, and reduce the time from research prototype to production deployment


Work environment:

This role is hybrid but requires occasional in-person work in Reston, VA.


Schedule: Monday – Friday. Occasional off-hours availability for infrastructure incidents, deployment windows, or training job monitoring.

Requirements:

Required qualifications:

• U.S. citizenship (required for work with federal customers)

• Willingness to obtain a TS/SCI clearance

• 5+ years of experience in infrastructure engineering, DevOps, or MLOps, with at least 2 years specifically supporting ML/AI workloads in production

• Deep hands-on experience with AWS

• Strong proficiency with infrastructure as code (Terraform required; CloudFormation, or CDK also valued)

• Experience building and managing Docker containers and container orchestration for ML workloads

• Production experience with GPU compute management - CUDA environments, NVIDIA driver management, multi-GPU configuration, and GPU memory optimization

• Strong scripting and automation skills in Python and Bash

• Experience with CI/CD systems (GitHub Actions, GitLab CI, or equivalent) including automated testing, build, and deployment pipelines

• Solid understanding of ML model lifecycle: training, evaluation, packaging, versioning, deployment, and monitoring

• Bachelor’s degree in Computer Science, Systems Engineering, or a related technical field (or equivalent professional experience)


Preferred qualifications:

• Familiarity with generative AI model serving – specifically diffusion or GAN-based models

• Background in air-gapped or disconnected environment deployments (NIST 800-171, CUI compliance, classified networks)

• Familiarity with ML experiment tracking tools (Weights & Biases, MLflow, or similar)

• Experience with cost optimization for GPU compute - spot instances, reserved capacity, right-sizing, and usage monitoring

• Prior work supporting federal law enforcement, intelligence community, or defense infrastructure

• Familiarity with Electron apps, or other desktop apps