Director of Infrastructure Engineering - AI

JPMorgan Chase & Co. • New York, NY

2d • $161,500 - $260,000

About The Position

The Director of Infrastructure Engineering leads one large technical area or multiple smaller areas within JPMorganChase’s Core Engineering Solutions and Architecture group, using advanced technical knowledge and independent judgment to resolve complex issues. The role sets the multi-year strategy and roadmap for AI infrastructure across Consumer & Community Banking, aligned with business priorities and the firm’s technology direction. It also establishes reuse-first expectations and governance standards for enterprise-authorized AI adoption and AI-assisted workflows, with appropriate human validation, data handling, traceability, security, resilience, and auditability.

The Director manages operational plans, budgets, staffing, and outcomes while building and developing a high-performing organization of engineers and engineering managers. The role works with business, product, technology, security, risk, finance, and other senior stakeholders to translate AI use cases into infrastructure requirements, influence shared technology roadmaps, and deliver production outcomes such as measurable cost savings and improved reliability. The position calls for deep infrastructure expertise, production experience with ML or GenAI workloads, and executive-level communication and stakeholder management.

Requirements

• Formal training or certification in infrastructure engineering concepts and 10+ years of applied experience.
• Experience managing cross-functional infrastructure engineering teams and a demonstrated record of setting technical strategy and delivering measurable outcomes.
• Experience leading teams in the safe use of enterprise-authorized AI capabilities for infrastructure engineering workflows, including validation practices and awareness of data sensitivity.
• Ability to define review and approval expectations for AI-assisted recommendations and ensure outcomes meet resiliency, security, and auditability expectations.
• Experience hiring, developing, and recognizing talent and creating a culture of ownership and engineering excellence.
• Extensive infrastructure engineering hardware knowledge, with a deep background in cloud infrastructure, platform, or site reliability engineering and hands-on depth with at least one major cloud provider; AWS examples include compute, networking, IAM, managed Kubernetes, and observability.
• Experience translating business needs into technical roadmaps and influencing central or shared technology teams not directly managed.
• Strong collaboration across engineering, product, security, risk, and finance.
• Excellent stakeholder management and executive communication, written and verbal, including presentations to senior leadership.
• Budget and vendor management experience and comfort making investment and cost trade-off decisions.
• Experience running ML or GenAI workloads in production, such as hosted model APIs behind API gateways with authentication, rate limiting, and resilience, and building AI skills, plugins, and agents.
• Experience with Kubernetes, including GPU scheduling and auto-scaling; infrastructure as code with Terraform or equivalent; and infrastructure CI/CD.
• Python and automation fluency to convert manual effort into automated solutions, together with observability expertise using tools such as Splunk, Datadog, Prometheus/Grafana, or OpenTelemetry.

Nice To Haves

• Experience with GPU compute orchestration, scheduling, and reclaim policies.
• Model serving and optimization experience, including AI Gateways, vLLM, Triton, TensorRT, batching, quantization, latency, and throughput.
• Distributed training experience with PyTorch FSDP, DeepSpeed, or Megatron-LM, and high-speed GPU interconnects such as NVLink or InfiniBand.
• Experience with RAG and vector databases such as OpenSearch, pgvector, or Pinecone, and agent frameworks such as LangChain or LlamaIndex.
• GPU capacity planning and FinOps experience for AI workloads, including cost allocation, chargeback, and reserved capacity.
• Experience leading an LLM gateway, AI platform, or ML platform organization.
• Leadership experience in financial services, including working with risk, compliance, and audit teams.

Responsibilities

• Manage technical area(s) within infrastructure engineering and collaborate across technical areas, anticipating the needs of multiple disciplines.
• Set the multi-year strategy and roadmap for AI infrastructure across Consumer & Community Banking, aligned with business priorities and the firm’s technology direction.
• Set reuse-first expectations for enterprise-authorized AI adoption, including human-in-the-loop validation and appropriate handling of sensitive data.
• Manage and implement corporate and divisional operational plans; develop and execute function objectives and provide direction to teams, team leads, and supervisors.
• Build, lead, and grow a high-performing organization of engineers and engineering managers through hiring, coaching, performance management, and succession planning.
• Manage senior stakeholders and ensure teams meet compliance, risk, security, service-level, and business requirements.
• Partner with business and product leaders to translate AI use cases into infrastructure requirements prioritized by value and risk.
• Represent business needs to firm-wide technology and platform teams, shape roadmaps, negotiate priorities and timelines, and escalate missed commitments.
• Drive large, complex projects, processes, and initiatives through to delivered outcomes, including production use cases, measurable cost savings, and improved reliability.
• Manage staffing, budgets, and relevant profit and loss; develop and manage budgets.
• Establish governance standards for AI-assisted delivery and operational workflows, ensuring traceability, auditability, resiliency, security, and risk alignment.

Benefits

• Competitive total rewards package, including base salary determined by role, experience, skill set, and location.
• Eligible roles may receive commission-based pay and/or discretionary incentive compensation in cash and/or forfeitable equity.
• Comprehensive health care coverage.
• On-site health and wellness centers.
• Retirement savings plan.
• Backup childcare.
• Tuition reimbursement.
• Mental health support.
• Financial coaching.

What This Job Offers

• Job Type: Full-time
• Career Level: Director
• Education Level: No Education Listed