Senior AI/ML Software Engineer

Peraton•Herndon, VA

About The Position

Peraton Labs is seeking a Senior AI/ML Software Engineer to join the Labs Agentic AI team. In this role, you'll design, build, and ship AI-powered systems, specifically a compliance-ready, low-code platform for dynamically generating and orchestrating AI agentic workflows. The work spans the full product lifecycle: from architecting multi-step agentic pipelines backed by Temporal.io, to building the plugin system, APIs, and interfaces that bring them to life, all within federal-grade security and accreditation constraints. This position is for someone who deeply considers how AI agents should operate in high-trust environments, prioritizes reliability and auditability, and can seamlessly transition between distributed orchestration, backend systems, and product-facing features. Peraton is a next-generation national security company that drives missions of consequence spanning the globe and extending to the farthest reaches of the galaxy. As the world’s leading mission capability integrator and transformative enterprise IT provider, Peraton delivers trusted, highly differentiated solutions and technologies to protect our nation and allies. Peraton operates at the critical nexus between traditional and nontraditional threats across all domains: land, sea, space, air, and cyberspace. The company serves as a valued partner to essential government agencies and supports every branch of the U.S. armed forces. Each day, our employees do the can’t be done by solving the most daunting challenges facing our customers.

Requirements

Minimum of a BS degree with 12 years of experience, MS degree with 10 years, or PhD with 7 years with meaningful exposure to AI/ML systems or LLM-based products
Hands-on experience building agentic systems using multi-step reasoning, tool use, RAG pipelines, or autonomous task execution
Strong Python skills (3.12+); comfort with async/await patterns, type hints, and modern Python tooling
Experience with workflow or task orchestration systems (Airflow, Prefect, Celery, or similar distributed execution frameworks)
Familiarity with agentic frameworks and an understanding of the underlying concepts (chains, tool calling, agent loops) that transfer across tools
Experience working with LLM APIs (OpenAI, Anthropic, AWS Bedrock, or similar)
Comfort working across the stack: FastAPI/Python backends, React frontends, Docker containerization, and PostgreSQL
A product mindset: you think about the end user, not just the technical implementation
Comfort operating with some ambiguity in a fast-moving environment
US Citizenship is a requirement for this position

Nice To Haves

Experience with workflow orchestration frameworks for workflow/activity patterns, task queues, worker lifecycle management
Familiarity with federal compliance environments: FedRAMP, FIPS 140-2/3, IronBank container hardening, OPA policy enforcement, or Section 508 accessibility
Experience building plugin or extension systems: dynamic code loading, container isolation, API mixin patterns
Exposure to orchestration patterns: supervisor agents, parallel tool calls, human-in-the-loop flows, DAG-based pipeline execution
Experience with observability tooling: OpenTelemetry, Jaeger, Prometheus, Grafana, or similar distributed tracing/metrics stacks
Familiarity with prompt engineering, evaluation frameworks, or agent observability
Experience with container orchestration (Docker SDK, Kubernetes) and distributed storage (S3, MinIO, JuiceFS)
Prior work building internal tooling, enterprise automation products, or platforms for government customers

Responsibilities

Design and implement agentic AI capabilities using Python-based frameworks (LangChain, LangGraph, DeepAgents) and orchestrated workflows
Build and maintain integrations with LLM APIs (Anthropic/Claude, OpenAI, AWS Bedrock, Ollama) to power intelligent, multi-step automations
Develop full-stack product features (FastAPI + React) that surface AI capabilities to users — from REST APIs and streaming interfaces to workflow builders and dashboards
Instrument agent pipelines with OpenTelemetry tracing, provenance audit trails, and observability tooling for debugging and performance evaluation
Write clear, well-tested, maintainable code that passes strict pre-commit validation, and contribute to engineering standards in a compliance-driven environment
Evaluate agent performance, debug distributed workflows, and continuously improve reliability and output quality