Senior LLM/AI Platform Engineer

Peraton•Annapolis Junction, MD

About The Position

Peraton Labs is seeking a senior engineer to serve as the technical lead for the multi-agent generative AI orchestration layer that powers an advanced analytical platform. This role owns the architecture governing how multiple LLM-based worker agents are spawned, routed, coordinated, and managed across diverse analytical tasks. The Senior LLM/AI Platform Engineer ensures that the platform's generative AI backbone is reliable, performant, and extensible as new worker types and capabilities are introduced.

Requirements

Minimum of 8+ years of experience with a Bachelor's degree, 6+ years with a Master's degree, or 3+ years with a PhD in Computer Science, AI/ML, or related field
Deep experience with large language model architectures, fine-tuning, and inference optimization
Proficiency in Python; experience with LLM frameworks (LangChain, LlamaIndex, or equivalent)
Strong understanding of prompt engineering, retrieval-augmented generation (RAG), and agent-based architectures
Experience with vector databases and embedding models
Demonstrated ability to architect complex, multi-component generative AI systems
US Citizenship with the ability to obtain/maintain a Secret clearance

Nice To Haves

Experience with multi-agent systems or autonomous agent frameworks
Familiarity with government/defense program management concepts
Experience with model evaluation and red-teaming methodologies
Knowledge of FedRAMP or DoD cloud security requirements
7+ years of software engineering experience, with 3+ years focused on LLM/NLP systems
Graduate degree in Computer Science, AI/ML, or related field is highly desired

Responsibilities

Architect and maintain the multi-agent orchestration framework, including worker spawning, conversation history isolation, context management, and inter-worker coordination
Design and implement token optimization strategies, including context window management, conversation summarization, and efficient prompt construction
Own the model routing layer that directs tasks to appropriate model tiers based on task complexity and user configuration
Develop and refine prompt engineering patterns for worker agents to ensure consistent, high-quality analytical outputs
Establish evaluation frameworks to measure generative AI output quality, accuracy, and reliability across worker types
Lead the design of new worker types to support program management capabilities (cost analysis workers, risk analysis workers, data fusion workers)
Collaborate with PM Domain Analysts to translate program management requirements into generative AI agent behaviors
Mentor junior LLM/AI Platform Engineers and establish engineering standards for generative AI development