Enterprise AI Orchestration Architect, Senior Manager

Booz Allen Hamilton•Washington, DC

21d•$162,800 - $303,000

About The Position

Build reusable agents and intelligent applications that perform enterprise work reliably. Develop agent behavior, orchestration, tool integrations, and evaluation capabilities to support customer solutions from demonstration through production. Translate workflows into tested software using model APIs, retrieval, tool calling, memory, and human oversight. Instrument execution, maintain evaluation datasets, and improve quality through trace analysis and regression testing. Partner with Platform, Data, and Forward-Deployed Engineering to deliver reusable components, integration contracts, and deployment guidance. Own agent behavior and application quality, including the evidence needed to validate them.

Requirements

10+ years of experience building maintainable applications, APIs, and automated tests using version control and software engineering practices.
Proficiency in Python or TypeScript.
Experience integrating large language models into applications, including prompt and context management, structured outputs, and tool calling.
Experience implementing agent orchestration, workflow state, retrieval-augmented generation, and enterprise API integrations.
Experience instrumenting model calls, tool execution, state transitions, and outcomes with logs or distributed traces.
Experience developing evaluation datasets and regression tests for task success, groundedness, safety, tool correctness, latency, and cost.
Ability to design appropriate human approval steps, tool permissions, input and output validation, error handling, and fallback behavior.
Ability to diagnose application failures and distinguish agent behavior defects from data, platform, or customer configuration issues.
Ability to document interfaces and limitations and collaborate on customer readiness and reusable engineering improvements.
Bachelor’s degree.

Nice To Haves

Experience with LangGraph, Semantic Kernel, MCP, OpenTelemetry, LangSmith, Langfuse, or comparable technologies.
Experience with multi-agent coordination, memory design, reusable SDKs, or reference architectures.
Experience delivering full-stack AI applications or improving production workflows through evaluation and trace analysis.

Responsibilities

Build reusable agents and intelligent applications for enterprise work.
Develop agent behavior, orchestration, tool integrations, and evaluation capabilities for customer solutions.
Translate workflows into tested software using model APIs, retrieval, tool calling, memory, and human oversight.
Instrument execution, maintain evaluation datasets, and improve quality through trace analysis and regression testing.
Partner with Platform, Data, and Forward-Deployed Engineering to deliver reusable components, integration contracts, and deployment guidance.
Own agent behavior and application quality, including the evidence needed to validate them.