Staff Enterprise AI Engineer

Peloton•New York, NY

10d•$193,550 - $237,750•Hybrid

About The Position

Peloton is looking to transform its enterprise tech strategy with AI adoption. The company is seeking a Staff Enterprise AI Engineer to serve as the "Founding Engineer" of their Enterprise AI Platform. This role is focused on architecting and building the operating system that enables Product, People, and Operations teams to deploy AI Agents safely and at scale. The engineer will act as a "Player/Coach," establishing the technical foundation (Infrastructure, Security, Orchestration) while guiding a team to execute the vision and build the "Golden Path" for leveraging AI securely.

Requirements

10+ years of software engineering experience, with 3+ years specifically focused on MLOps, LLM Orchestration, or Large Scale Distributed Systems.
Deep fluency in Python (production grade) and Go (preferred for platform services).
Proven experience deploying RAG (Retrieval Augmented Generation) and Agentic Workflows in production.
Experience with frameworks like LangChain, Semantic Kernel, or similar.
Strong background in Kubernetes (EKS), Docker, and Infrastructure-as-Code (Terraform).
Solid understanding of OAuth 2.0 (OBO flow), RBAC, and zero-trust networking principles.
Ability to explain complex technical trade-offs (e.g., "Latency vs. Accuracy") to executive stakeholders.

Nice To Haves

Experience implementing Model Context Protocol (MCP) or similar standardized tool interfaces.
Background in FinOps (managing GPU/Cloud spend).
Experience navigating highly regulated environments (HIPAA, SOX, etc.).

Responsibilities

Architect the "Intelligence & Integration" Layers, designing and building a scalable Agentic Orchestration Platform (using LangChain, LangGraph, or custom frameworks) that allows internal developers to spin up autonomous agents.
Implement the "Integration Layer" ensuring all AI agents connect to internal APIs (Workday, Snowflake, SAP) via secure, standardized protocols (Model Context Protocol - MCP).
Solve the "State Problem" for AI, architecting memory stores (Vector DBs like Pinecone/Weaviate) that persist context across user sessions.
Enforce "Security by Design" by partnering with Security leadership to implement Identity Propagation, ensuring agents execute tasks using the user’s specific OAuth scopes, preventing privilege escalation.
Build "Data Clean Rooms" and PII masking pipelines to ensure sensitive member or employee data is never leaked to model providers.
Deploy EvalOps pipelines to automatically test models for hallucination and regression before they hit production.
Define the Engineering Standards, including "Guide vs. Control" standards, and create templates and libraries for safe "Vibe Code" (low-code/assisted coding).
Perform rigorous code reviews for partner teams and vendors, ensuring high performance, low latency (<200ms), and cost efficiency.
Optimize inference costs by implementing Semantic Caching and routing logic (e.g., routing simple queries to smaller/cheaper models).
Leverage Kubernetes (EKS) to manage ephemeral compute resources for AI workloads.
Mentor senior engineers and demystify AI for non-technical stakeholders.