Head of Agent Engineering
About The Position
Baker Tilly is seeking a Head of Agent Engineering to serve as the senior technical leader responsible for defining how the firm’s AI agents are designed, orchestrated, evaluated, and deployed. The role establishes the standards and patterns product teams use for task decomposition, orchestration, context and memory management, tool use, human-in-the-loop controls, and failure recovery. The Head of Agent Engineering owns the evaluation harness and baselines used to guide iteration and inform release and acceptance decisions for internally developed and third-party agents. The role also has end-to-end ownership of BT Compass, the firm’s expert front door, including its agentic workflows and product experiences, built on the intelligence layer developed by the Data & Intelligence team. As the Agent Factory grows, this leader will build and mentor the team, establish engineering and design standards, and partner with internal teams and external experts to advance Baker Tilly’s agentic engineering capabilities. This is a hands-on engineering leadership role involving direct participation in code and design reviews and the translation of emerging agentic engineering practices into scalable, firm-owned standards.
Requirements
- Bachelor’s degree in computer science, engineering, or a related field, or equivalent practical experience.
- 15+ years of software engineering experience, including 3+ years building and deploying LLM-based or agentic systems in production.
- Staff- or principal-level engineering experience, with demonstrated success personally designing and shipping production agent systems.
- Deep experience designing and building evaluation harnesses and using evaluation results, baselines, and regression testing to drive iteration and release decisions.
- Strong knowledge of the modern agentic technology stack, including orchestration frameworks, tool/function calling and MCP, retrieval using vector and graph approaches, and context and memory management.
- Experience with evaluation methodologies, including evaluation design, baselines, regression discipline, confidence frameworks, and LLM-as-judge patterns and limitations.
- Full-stack product engineering experience, including building the applications and user experiences in which agents operate.
- Experience designing human-in-the-loop workflows, checkpoints, review experiences, and failure-containment mechanisms.
- Strong hands-on software engineering skills, with recent production-quality coding experience in Python and/or TypeScript.
- Experience with observability and tracing for agentic systems.
- Experience directing third-party or partner engineers against established technical standards.
- Experience evaluating third-party technologies or vendor agents against internal technical and quality standards.
- Demonstrated ability to use AI and agentic development tools as a force multiplier in day-to-day engineering work.
- Strong ability to collaborate across engineering, product, design, platform, and business teams and translate emerging technical practices into repeatable standards.
- No certifications or licenses required.
Nice To Haves
- Advanced degree preferred.
Responsibilities
- Define and build agentic reference patterns, including task decomposition, orchestration, context engineering, memory management, tool use, human-in-the-loop checkpoints, and failure containment and recovery.
- Maintain the agent pattern library and establish orchestration standards for product teams.
- Own BT Compass end to end, including its agentic workflows and full-stack product experiences, on the intelligence layer built by the Data & Intelligence team.
- Build and own the evaluation harness, baselines, and regression discipline used to drive iteration and inform release and acceptance decisions for internally developed and third-party agents.
- Establish evaluation and acceptance criteria in partnership with Product, Architecture, Platform & Production Engineering, Quality Engineering, and other stakeholders.
- Partner with Platform & Production Engineering to enable evaluation capabilities to operate at scale in production.
- Lead the growing Agent Factory and set standards for agent design, engineering quality, and full-stack product development.
- Conduct hands-on design and code reviews across product teams and mentor engineers in agentic engineering practices.
- Hire and develop full-stack, forward-deployed AI engineers who can build agents and the applications in which they operate.
- Partner with agentic-engineering experts to adopt leading practices and translate them into Baker Tilly-owned standards, patterns, and runbooks.
- Direct third-party and partner engineers delivering solutions against Baker Tilly’s technical standards.
- Evaluate third-party and vendor agents against the firm’s technical and evaluation standards.
- Partner with the Chief Architect to establish evaluation standards and promote proven patterns into the platform’s shared library.
- Collaborate with Data & AI Platform, Product, Design, Quality Engineering, Platform & Production Engineering, subject matter experts, and engineering teams in India.
- Engage in special projects and perform other duties as assigned.
Benefits
- Compensation and benefits package offered to eligible employees.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
