Executive Director, Product Director – GenAI Gateway & Orchestration

JPMorgan Chase & Co.•Palo Alto, CA
•$180,500 - $285,000

About The Position

As a Product Director in the Model & Serving team, you lead the development of customer-centric products and features that shape how enterprise teams access AI models. You define and deliver a runtime platform with routing, guardrails, observability, telemetry, evaluation, and cost management built in, helping teams move from idea to production while meeting enterprise requirements for security, compliance, multi-tenancy, reliability, and cost control. The role calls for a senior product leader with expertise in AI infrastructure, distributed systems, and model serving. In partnership with Engineering, Architecture, Cybersecurity, Risk, Finance, and business teams, you set the product vision and target-state gateway architecture, lead execution across federated and managed deployment models, and ensure the platform scales reliably as demand grows. You also coach product team members and communicate strategy, progress, risks, dependencies, and recommendations to senior leadership and governance forums.

Requirements

  • 8+ years of experience or equivalent expertise delivering products, projects, or technology applications.
  • Extensive knowledge of the product development life cycle, technical design, and data analytics.
  • Proven ability to influence adoption of product life cycle activities, including discovery, ideation, strategic development, requirements definition, and value management.
  • Experience driving organizational change and managing stakeholders across multiple functions.
  • Experience building or scaling AI or cloud platform infrastructure, such as generative AI or large language model gateways, API gateways and API management, model serving platforms, cloud networking and traffic management, or enterprise observability and telemetry platforms.
  • Experience building AI and machine learning platform products for enterprise customers on one or more major cloud providers (Amazon Web Services, Microsoft Azure, or Google Cloud), including compute, networking, storage, and security for AI workloads.
  • Strong knowledge of generative AI architectures, large language models, and how AI applications consume models in production.
  • Understanding of AI infrastructure and model serving, including inference performance, capacity planning, scalability, and reliability engineering.
  • Experience with cost management for cloud or AI workloads, including usage metering, cost attribution, and optimization.
  • Ability to translate complex technical concepts into clear product strategy, prioritized roadmaps, and executive communications.
  • Bachelor's or Master's degree in Computer Science, Engineering, or a related technical field, or equivalent practical experience.

Nice To Haves

  • Recognized thought leader within a related field.
  • Experience building or leading a generative AI gateway, large language model proxy, model routing, or machine learning serving product at enterprise scale.
  • Full-stack technical fluency across the AI platform, from GPU and accelerator infrastructure and low-latency serving stacks to APIs, SDKs, and the applications built on them.
  • Experience operating both federated and centrally managed platform models, including shared standards, policy enforcement, and multi-tenant design.
  • Familiarity with open-source and commercial gateway and model serving technologies, as well as techniques for optimizing inference performance and cost.
  • Familiarity with AI observability, evaluation, and guardrail tooling, and with agent orchestration.
  • Experience partnering closely with Legal, Compliance, and Risk teams, with working knowledge of the regulatory landscape for AI, data privacy, and third-party risk in financial services.

Responsibilities

  • Oversee the product roadmap, vision, development, execution, risk management, and business growth targets.
  • Lead the product life cycle from planning and execution through future development, adapting methodologies, managing risks, and achieving targets for cost, features, reusability, and reliability.
  • Coach and mentor the product team on solution generation, market research, storyboarding, mind-mapping, prototyping, adoption strategies, and product delivery.
  • Own product performance and invest in enhancements to achieve business objectives.
  • Monitor market trends, conduct competitive analysis, and identify product differentiation opportunities.
  • Serve as the accountable product leader for the generative AI Gateway across federated and managed deployment models, driving adoption and platform outcomes.
  • Define the target-state gateway architecture with Engineering and Architecture partners, including data, multi-cloud, and multi-region deployment strategies and consistent capabilities across deployment models.
  • Lead token management and intelligent model routing across policy, cost, and latency for multi-model and agentic workloads.
  • Drive traffic management and resiliency so the platform scales reliably, partnering with Engineering to optimize inference performance and cost.
  • Make security and governance default capabilities for every model call, and own guardrails, observability, and telemetry that provide visibility into production behavior.
  • Communicate strategy, delivery progress, risks, dependencies, and recommendations to senior leadership and governance forums.

Benefits

  • Competitive total rewards package, including base salary determined by role, experience, skill set, and location.
  • Eligible roles may receive commission-based pay and/or discretionary incentive compensation in cash and/or forfeitable equity.
  • Comprehensive health care coverage.
  • On-site health and wellness centers.
  • Retirement savings plan.
  • Backup childcare.
  • Tuition reimbursement.
  • Mental health support.
  • Financial coaching.
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service