Enterprise AI Orchestration Architect, Senior Manager

Booz Allen Hamilton•McLean, VA

21d•$162,800 - $303,000•Hybrid

About The Position

The Enterprise AI Orchestration Architect will build reusable agents and intelligent applications that reliably perform enterprise work. The role develops agent behavior, orchestration, tool integrations, and evaluation capabilities for customer solutions from demonstration through production. The architect translates workflows into tested software using model APIs, retrieval, tool calling, memory, and human oversight. They instrument execution, maintain evaluation datasets, and improve quality through trace analysis and regression testing. In partnership with Platform, Data, and Forward-Deployed Engineering, they deliver reusable components, integration contracts, and deployment guidance. They own agent behavior and application quality, including the evidence needed to validate them.

Requirements

10+ years of experience building maintainable applications, APIs, and automated tests using version control and software engineering practices.
Proficiency in Python or TypeScript.
Experience integrating large language models into applications, including prompt and context management, structured outputs, and tool calling.
Experience implementing agent orchestration, workflow state, retrieval-augmented generation, and enterprise API integrations.
Experience instrumenting model calls, tool execution, state transitions, and outcomes with logs or distributed traces.
Experience developing evaluation datasets and regression tests for task success, groundedness, safety, tool correctness, latency, and cost.
Ability to design appropriate human approval steps, tool permissions, input and output validation, error handling, and fallback behavior.
Ability to diagnose application failures and distinguish agent behavior defects from data, platform, or customer configuration issues.
Ability to document interfaces and limitations and collaborate on customer readiness and reusable engineering improvements.
Bachelor’s degree.

Nice To Haves

Experience with LangGraph, Semantic Kernel, MCP, OpenTelemetry, LangSmith, Langfuse, or comparable technologies.
Experience with multi-agent coordination, memory design, reusable SDKs, or reference architectures.
Experience delivering full-stack AI applications or improving production workflows through evaluation and trace analysis.

Responsibilities

Build reusable agents and intelligent applications for reliable enterprise work.
Develop agent behavior, orchestration, tool integrations, and evaluation capabilities for customer solutions.
Translate workflows into tested software using model APIs, retrieval, tool calling, memory, and human oversight.
Instrument execution and maintain evaluation datasets.
Improve quality through trace analysis and regression testing.
Partner with Platform, Data, and Forward-Deployed Engineering to deliver reusable components, integration contracts, and deployment guidance.
Own agent behavior and application quality, including the evidence needed to validate them.