Python AI Engineer

EPAM Systems•New York, NY

About The Position

The Python AI Engineer will operate and evolve the infrastructure behind an enterprise knowledge base platform as it develops from a Confluence-focused RAG chatbot into a broader agentic knowledge system. The current platform supports Confluence and GitHub ingestion, chunking, pgvector, RAG retrieval, and FastAPI serving. The next phase includes hybrid retrieval (vector, sparse, and graph), ingestion from multiple sources, robust evaluation pipelines, agent infrastructure, and shared chat platform primitives. The role focuses on building scalable, reliable, production-grade AI infrastructure. It involves backend and agent-runtime development, retrieval and system design trade-offs, observability, operational feedback, and dependable AI behavior across multiple conversational surfaces.

Requirements

Strong hands-on Python experience in platform, automation, or infrastructure-heavy environments.
Experience building CLI tools using Python, Golang, or Rust.
Hands-on experience with LangGraph, LangChain, pgvector, and modern retrieval pipelines.
Experience designing evaluation frameworks for LLM-backed systems, including regression detection and quality measurement.
Strong experience with Docker, Helm, GitHub Actions, and Kubernetes-oriented workflows.
Familiarity with the operational characteristics of embedding pipelines, vector search, and LLM-backed systems.
Strong skills in metrics, tracing, dashboards, alerting, and log analysis.
Experience with ingestion, ETL, or content-processing pipelines at scale.
Ability to consider reliability, cost, latency, throughput, and disaster recovery.

Nice To Haves

Experience with Qdrant, Neo4j, or other vector or graph infrastructure.
Prior experience supporting RAG, search, evaluation, or agent platforms.
Background in enterprise or highly regulated environments.
Familiarity with enterprise tooling such as Vault, Splunk, Artifactory, or ECR.
Comfort leveraging AI-assisted engineering workflows in day-to-day work.

Responsibilities

Architect and implement backend services using Python 3.11, FastAPI, Pydantic, SQLAlchemy async, and asyncpg.
Make design trade-offs involving retrieval quality, latency, cost, safety, and operational simplicity.
Build production-grade agent runtime capabilities, including memory boundaries, tool sandboxing, permissions, and budget controls.
Improve answer grounding, failure analysis, and citation enforcement for enterprise-grade dependability.
Create observability and operational feedback loops using OpenTelemetry, Prometheus/Grafana, Docker/Helm, and GitHub Actions.
Develop ingestion infrastructure for current and future content sources.
Provide observability across applications, pipelines, databases, and model-serving behavior.
Manage cost, latency, throughput, and failure modes for AI-heavy workloads.
Establish release workflows that validate changes to AI behavior, not just code compilation.
Partner with product and engineering teams to support multiple conversational surfaces through a unified knowledge platform.