Principal Engineer, Agent Platform

Gap Inc.•Coppell, TX

About The Position

We are building the enterprise AI agent platform that teams across the company will use to create, deploy, govern, and operate AI-powered solutions at scale. As a Principal Engineer, you will help define the technical foundation for agent development across the organization, building the services, frameworks, and guardrails that make AI agents secure, reliable, observable, and easy to adopt. This is a hands-on engineering role focused on architecture, software design, and platform development. You will work across AI, distributed systems, cloud infrastructure, security, and governance to deliver a platform that enables product and engineering teams to move quickly while meeting enterprise standards for security, compliance, and operational excellence.

Requirements

Extensive experience building and operating large-scale distributed systems in production
Strong software engineering background with expert-level Python development skills
Hands-on experience delivering production LLM, generative AI, or agent-based solutions
Deep understanding of agent architectures, orchestration patterns, and enterprise AI platforms
Experienced with retrieval systems, embeddings, vector search, and context management strategies
Knowledgeable in AI evaluation, testing methodologies, and quality assurance for non-deterministic systems
Proficient with Google Cloud services including Vertex AI, Gemini, GKE, Cloud Run, BigQuery, and Pub/Sub
Experienced with cloud security, workload identity, OAuth, policy-as-code, and secure-by-design architectures
Strong understanding of distributed system reliability patterns, including resiliency, retries, observability, and fault tolerance
Proven ability to influence technical direction, drive adoption of architectural standards, and lead through expertise rather than authority

Responsibilities

Design and build core platform services that enable enterprise AI agent development and operations
Define the architecture, engineering patterns, and technical standards for the AI agent platform
Develop and maintain a centralized model gateway for routing, governance, resiliency, and cost management
Build agent runtime capabilities that support long-running workflows, orchestration, and human-in-the-loop processes
Design secure identity, authorization, and policy enforcement mechanisms for agents and tools
Establish platform capabilities for context management, memory, retrieval, and data isolation
Create testing, evaluation, and release frameworks that ensure agent quality, safety, and performance
Implement observability, auditability, and operational tooling for monitoring and troubleshooting agent behavior
Partner with Security, Data Governance, and Cloud Infrastructure teams to define platform controls and guardrails
Mentor engineers, influence technical direction, and raise engineering standards across multiple teams