AI Architect

Navan • New York, NY

1d • $129,600 - $288,000

About The Position

The AI Architect will set technical direction for how the company designs, evaluates, integrates, and operates AI systems that improve important human workflows. The role guides teams in choosing appropriate approaches—from conventional software and deterministic workflows to retrieval-augmented systems and agents—and ensures systems meet standards for quality, reliability, safety, and value.

This is a strategic, hands-on position focused on establishing reusable architecture and evaluation practices, guiding cross-functional teams, and prototyping or validating key patterns. The AI Architect partners with product, engineering, operations, and domain leaders to turn high-value opportunities into production systems whose outcomes can be measured and operated.

Requirements

• At least 8 years of experience in backend software development.
• At least 3 years of experience in AI architecture with demonstrated organization-wide impact.
• Strong experience designing and delivering production software systems, including substantial work on AI/ML-powered products or services.
• Track record of setting technical direction across teams and influencing decisions through evidence and collaboration.
• Practical understanding of LLM applications, including retrieval and context construction, tool use, agent patterns, model selection, and failure handling.
• Experience defining quality expectations and building evaluations for AI behavior, and connecting offline evaluations to production outcomes.
• Sound judgment about when AI is appropriate and when a simpler deterministic approach is better.
• Experience with production concerns such as observability, privacy and security, reliability, cost, latency, and safe human escalation.
• Clear communication across engineering, product, and executive audiences; a builder's mindset and comfort prototyping and reviewing implementation.

Nice To Haves

• Experience establishing shared AI platforms, service patterns, or architecture standards across multiple teams.
• Experience creating evaluation datasets, test harnesses, regression suites, or quality dashboards for LLM or agent systems.
• Experience operating AI products with measurable user or business outcomes.
• Experience working with cloud AI platforms, model APIs, agent frameworks, tracing and evaluation tools, and data platforms.

Responsibilities

• Define AI reference architectures and design principles for AI services, agents, model integrations, data and context flows, tools, and human review; balance quality, autonomy, reliability, latency, cost, and complexity.
• Establish evaluation practices, including expected behaviors, success measures, representative datasets, evaluation methods, regression checks, quality thresholds, and ongoing evaluation maintenance.
• Shape shared production-readiness practices for tracing and observability, versioning, monitoring, incident response, privacy, security, and lifecycle management.
• Create reusable patterns, reference implementations, and practical guidance; coach engineers through architecture reviews and help teams move valuable work from prototype to production.
• Partner with product, operations, and domain teams to identify workflow needs and measure AI's effect on completion, quality, time, or other agreed outcomes.
• Prototype high-risk or foundational designs, review critical implementations, and refine architecture using production evidence.
• Evaluate models, agent runtimes, orchestration approaches, and evaluation tools against requirements, and recommend adoption based on evidence.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed