Senior LLMOps Engineer

Steampunk • McLean, VA

11d • $145,000 - $185,000

About The Position

We are looking for an experienced Senior LLMOps Engineer to design, implement, and maintain production-grade large language model (LLM) pipelines, deployment architectures, and monitoring systems across enterprise environments. The Senior LLMOps Engineer will play a critical role in operationalizing generative AI capabilities, ensuring that LLM-based applications are scalable, secure, reliable, and compliant with emerging AI risk and governance frameworks. This role spans model deployment, orchestration, evaluation, optimization, and Retrieval-Augmented Generation (RAG) pipelines. You will contribute to the growth of our AI & Data Exploitation Practice!

Requirements

• Ability to obtain and maintain a U.S. government security clearance.
• Bachelor’s degree and 10+ years of total experience.
• 5+ years of experience in software engineering, data engineering, MLOps, or cloud engineering, with 2+ years focused specifically on LLM or GenAI operations.
• Hands-on experience designing, building, and maintaining Retrieval-Augmented Generation (RAG) pipelines.
• Fluency in Python.
• Strong experience deploying models using frameworks such as Hugging Face Transformers, vLLM, TensorRT-LLM, or similar.
• Experience with operational tooling and frameworks such as FastAPI, PyTorch, LangChain, LlamaIndex, and vector databases such as FAISS, Milvus, Pinecone, or similar.
• Advanced knowledge of cloud platforms such as AWS, Azure, or GCP, including model hosting, distributed compute, and secure networking patterns.
• Hands-on experience building CI/CD pipelines, automated testing frameworks, and environment provisioning for AI/ML workloads.
• Experience with Docker, Kubernetes, and infrastructure-as-code tools such as Terraform or CloudFormation.
• Familiarity with MLSecOps, AI governance, model hardening, prompt injection defenses, and content safety monitoring.
• Strong understanding of logging, observability, and performance profiling for high-throughput LLM inference systems.
• Excellent written and verbal communication skills, with the ability to explain trade-offs and architectural decisions to technical and non-technical stakeholders.
• Demonstrated ability to balance long-term platform thinking with hands-on operations and rapid problem solving.
• Experience working in Agile development environments and using modern project management tools.

Responsibilities

• Architect, build, and maintain scalable LLM and Retrieval-Augmented Generation (RAG) pipelines, including model hosting, inference optimization, retrieval layers, and context management frameworks.
• Lead the design and implementation of secure GenAI infrastructure across cloud environments, ensuring reliability, performance, scalability, and cost efficiency.
• Build and manage automated evaluation systems that assess LLM output quality, safety, latency, and adherence to AI governance requirements.
• Develop CI/CD workflows tailored for LLM- and GenAI-based applications, including dataset versioning, model lineage, and automated testing of prompt and model behaviors.
• Collaborate with AI Product Engineers and Data Scientists to productionize LLM-based prototypes into enterprise-grade, maintainable systems.
• Integrate vector databases, model gateways, content filters, and guardrail frameworks into end-to-end LLM and RAG solutions.
• Implement observability and monitoring solutions that track performance metrics, hallucination rates, cost profiles, and user interaction patterns.
• Lead troubleshooting and root-cause analysis for issues related to LLM deployment, inference performance, retrieval pipelines, or overall pipeline reliability.
• Develop and maintain Python-based services, integrations, and automation supporting LLM and RAG pipelines.
• Stay current with emerging LLM architectures, inference optimizations, fine-tuning techniques, and relevant MLSecOps patterns.
• Ensure compliance with data privacy, ethical AI, security, and AI governance frameworks throughout pipeline design and operations.
• Mentor junior engineers and contribute to AI engineering best practices, tooling, and reusable infrastructure patterns.

Benefits

• The projected compensation range for this position is $145,000 to $185,000.
• Annual salary is just one aspect of Steampunk’s total compensation package for employees.
• Learn more about additional Steampunk benefits here.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior