Lead Python AI Engineer

EPAM Systems • New York, NY

2d

About The Position

Build robust, AI-powered API services and agentic workflows, bridging traditional backend systems with modern language models. The role focuses on secure and scalable LLM interactions with internal systems, vector databases, and external tools through orchestration frameworks. LLMs are treated as components in a secure, scalable distributed system, with emphasis on API reliability, graceful failure handling, and production-grade AI integration.

The position owns API contracts and service reliability end to end, integrates LLMs with rigorous systems engineering, and builds reusable orchestration frameworks for downstream developers. It also contributes to architecture decisions and engineering standards while collaborating with frontend, data, and infrastructure teams.

Requirements

• 7+ years of backend engineering experience, with a heavy focus on microservices and distributed systems.
• Experience using Python for high-performance backend services and cloud-native APIs.
• Hands-on experience with AWS, Docker, ECS/EKS, and RESTful APIs.
• Proven experience with secure coding practices and strong authentication and authorization fundamentals.
• Practical, hands-on experience with agentic frameworks such as LangChain, LlamaIndex, or LangGraph.
• 1–2+ years of hands-on experience integrating AI SDKs such as OpenAI, Anthropic/Claude, AWS Bedrock, or equivalent in production environments.
• Familiarity with AI evaluation tooling or real-time APM platforms such as LangSmith, Langfuse, or Arize.
• Practical experience with vector databases such as Amazon Kendra or OpenSearch, advanced embedding strategies, and retrieval systems.

Nice To Haves

• 1–2+ years of experience with React and TypeScript.
• Experience deploying applications to EKS environments at scale.
• Knowledge of agent interoperability patterns such as MCP and identity/security domain models such as IAM and CIAM.
• Professional proficiency in Java, Node.js, or Go.

Responsibilities

• Own API contracts and service reliability end to end, ensuring AI edge cases fail gracefully without compromising system integrity.
• Integrate LLMs with evaluation, comprehensive observability, robust fallback handling, and cost awareness.
• Build reliable, reusable orchestration frameworks and logic for downstream developer consumption.
• Design and build scalable backend API services and advanced LLM orchestration layers.
• Implement and maintain complex RAG pipelines, including document ingestion, chunking, embedding, and retrieval optimization.
• Develop and connect agent tools using LangChain, LangGraph, and potentially MCP (Model Context Protocol).
• Ensure security, privacy, enterprise-grade observability, and comprehensive test coverage across backend workflows.
• Contribute to architecture decisions and engineering standards within the pod.
• Collaborate with frontend engineers, data engineers, and infrastructure teams.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed