Agentic AI Architect

EXL•Jersey City, NJ

About The Position

This role focuses on defining and leading the architecture for agentic AI-enabled platforms. The architect will be responsible for the end-to-end design across data, AI, orchestration, and integration layers, with hands-on experience in Proofs of Concept (POCs). Key responsibilities include designing and governing the agentic orchestration framework for production workflows, establishing architecture patterns for Retrieval Augmented Generation (RAG), vector search, retrieval, Model Context Protocol (MCP) tool access, and prompt management/evaluation. A deep understanding of agentic coding and its large-scale implementation is crucial, along with familiarity in implementing A2A or similar frameworks. The role also involves platform and integration design, including integration across various data systems and third-party APIs, designing a configurable framework for multi-Line of Business (LOB) onboarding, and defining API/microservices patterns. AI and GenAI enablement includes defining the appropriate use of GenAI versus deterministic logic and agentic workflows versus pipeline workflows, establishing multimodal integration, and designing prompt lifecycle, evaluation, and optimization strategies. Furthermore, the architect will define AI safety and guardrails, establish ModelOps and PromptOps frameworks, and ensure explainability, auditability, and traceability of AI outputs. Program leadership involves leading technical execution, guiding engineering teams, and driving technical decisions and stakeholder communication.

Requirements

GenAI & Agentic Frameworks - Semantic Kernel/ LangGraph (or similar orchestration frameworks)
LLM integration (Azure OpenAI, OpenAI APIs, etc.)
Prompt engineering, prompt lifecycle design
Retrieval & RAG - Azure AI Search (indexing, vector search, hybrid search)
Embedding pipelines and retrieval optimization
RAG design, grounding strategies, context management
Tool Access & Integration - MCP (Model Context Protocol) architecture and tool design
API design (FastAPI / REST / microservices)
Integration with enterprise systems and third-party APIs
AI Safety & Governance - NVIDIA NeMo Guardrails
Microsoft Presidio (PII detection/masking)
Guardrails for prompt injection, hallucination control
Evaluation & ModelOps - Azure AI Foundry (model hosting, versioning, monitoring)
Evaluation frameworks (LLM-as-judge, test datasets)
Prompt/version control, cost/latency monitoring
DevOps & Observability - CI/CD pipelines (Azure DevOps / GitHub Actions)
Logging, monitoring, observability (App Insights, etc.)
Performance tuning and scalability
Bachelor’s or Master’s in Computer Science, Engineering, Data Science, or related field

Responsibilities

Define end-to-end architecture for agentic AI-enabled platform across data, AI, orchestration, and integration layers with some real hands-on experience doing POCs
Design and govern agentic orchestration framework for multi-step production workflows
Establish architecture patterns for - RAG and grounding, Vector search and retrieval, MCP tool access layer, prompt management and evaluation
Have a deep understanding of Agentic coding and best practices of using Agentic coding for large scale implementations
Familiarity in implementing A2A or similar frameworks in a large scale environment
Define integration architecture across - Lakehouse, ODS, document systems, Underwriting systems and third-party APIs
Design configurable, metadata-driven framework for multi-LOB onboarding
Define API/microservices patterns (Python/.NET hybrid)
Define where and how to use - GenAI vs deterministic logic, agentic workflows vs pipeline workflows
Establish multimodal integration approach combining structured, unstructured, and external data
Design prompt lifecycle, evaluation, and optimization strategy
Define AI safety and guardrails (PII, hallucination control, policy constraints)
Establish ModelOps and PromptOps frameworks
Ensure explainability, auditability, and traceability of AI outputs
Lead technical execution across AI, data, and platform teams
Guide engineers (AI, data, full-stack) and ensure alignment with architecture
Drive technical decisions and stakeholder communication