About The Position

The Product Director in the Model & Serving team leads customer-centric product development for the firm’s generative AI Gateway and orchestration platform. The role defines how enterprise teams access models through a runtime layer that provides routing, guardrails, observability, telemetry, evaluation, and cost management by default. In partnership with Engineering, Architecture, Cybersecurity, Risk, Finance, and business teams, the Product Director delivers secure, resilient infrastructure that accelerates the path from idea to production while meeting enterprise requirements for security, compliance, multi-tenancy, and cost control. This senior product leader sets the platform vision and leads execution at enterprise scale, balancing speed and simplicity for application teams with reliability as demand grows. The role is accountable for product outcomes across federated and managed deployment models, including architecture, token management, intelligent model routing, traffic management, governance, and platform adoption.

Requirements

  • At least 8 years of experience, or equivalent expertise, delivering products, projects, or technology applications.
  • Extensive knowledge of the product development life cycle, technical design, and data analytics.
  • Ability to influence adoption of product life-cycle activities, including discovery, ideation, strategic development, requirements definition, and value management.
  • Experience driving organizational change and managing stakeholders across multiple functions.
  • Experience building or scaling AI or cloud platform infrastructure, such as generative AI or large language model gateways, API gateways and API management, model serving platforms, cloud networking and traffic management, or enterprise observability and telemetry platforms.
  • Experience building AI and machine learning platform products for enterprise customers on one or more major cloud providers (AWS, Microsoft Azure, or Google Cloud), including compute, networking, storage, and security for AI workloads.
  • Strong knowledge of generative AI architectures, large language models, and how AI applications consume models in production.
  • Solid understanding of AI infrastructure and model serving, including inference performance, capacity planning, scalability, and reliability engineering.
  • Experience with cloud or AI workload cost management, including usage metering, cost attribution, and optimization.
  • Ability to translate complex technical concepts into clear product strategy, prioritized roadmaps, and executive communications.
  • Bachelor's or Master's degree in Computer Science, Engineering, or a related technical field, or equivalent practical experience.

Nice To Haves

  • Recognized thought leader within a related field.
  • Experience building or leading a generative AI gateway, large language model proxy, model routing, or machine learning serving product at enterprise scale.
  • Full-stack technical fluency across the AI platform, from GPU and accelerator infrastructure and low-latency serving stacks to APIs, SDKs, and applications.
  • Experience operating both federated and centrally managed platform models, including shared standards, policy enforcement, and multi-tenant design.
  • Familiarity with open-source and commercial gateway and model serving technologies, and techniques for optimizing inference performance and cost.
  • Familiarity with AI observability, evaluation, and guardrail tooling, and agent orchestration.
  • Experience partnering closely with Legal, Compliance, and Risk teams, with working knowledge of AI regulation, data privacy, and third-party risk in financial services.

Responsibilities

  • Oversee product roadmap, vision, development, execution, risk management, and business growth targets.
  • Lead the full product life cycle, from planning and execution through future development, adapting approaches and managing risks to achieve targets for cost, features, reusability, and reliability.
  • Coach and mentor the product team in product discovery, solution generation, market research, storyboarding, mind-mapping, prototyping, adoption strategies, and delivery.
  • Own product performance and invest in enhancements that achieve business objectives.
  • Monitor market trends, conduct competitive analysis, and identify opportunities for product differentiation.
  • Serve as the accountable product leader for the generative AI Gateway across federated and managed deployment models, driving adoption and platform outcomes.
  • Define the target-state gateway architecture with Engineering and Architecture partners, including data, multi-cloud, and multi-region deployment strategies and consistent capabilities across deployment models.
  • Lead token management and intelligent model routing across policy, cost, and latency for multi-model and agentic workloads.
  • Drive traffic management and resiliency capabilities, partnering with Engineering to optimize inference performance and cost.
  • Make security and governance default capabilities for every model call, and own guardrails, observability, and telemetry that provide visibility into production behavior.
  • Communicate strategy, delivery progress, risks, dependencies, and recommendations to senior leadership and governance forums.
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service