Sr AI Technical Product Manager

T-Mobile•Frisco, TX

3d•$130,900 - $236,200•Hybrid

About The Position

The Sr AI Technical Product Manager is a product leader, customer evangelist, and execution driver responsible for taking T-Mobile's AI products from concept through beta, General Availability, and scale. This role owns product strategy and delivery for one or more AI products or platform capabilities, working in a fast-paced, entrepreneurial environment within T-Mobile's AI Commercialization team. The Sr AI Technical Product Manager combines customer empathy, product judgment, and hands-on technical execution with deep expertise in AI evaluation, including agentic testing and LLM-as-judge methods, so that every product meets carrier-grade standards for accuracy, latency, safety, and customer experience before and after launch. The role works closely with cross-functional teams to validate, build, and launch AI-powered solutions that create new revenue streams and differentiated customer experiences. This role manages the end-to-end delivery of complex technical AI products to meet specific business objectives and customer needs. It involves defining product vision, strategy, and roadmaps for an assigned product area while collaborating with cross-functional teams to deliver technical solutions. The role requires analyzing market data, customer insights, product telemetry, and evaluation results to prioritize product features, quality investments, and innovations. Success is measured by product performance, beta-to-GA conversion, customer satisfaction, production quality and evaluation scores, and alignment with business goals. The work impacts organizational growth and enhances customer experiences through effective product delivery and technical leadership.

Requirements

Bachelor's Degree - plus 7 years of related work experience OR Advanced degree with 5 years of related experience. Acceptable areas of study include Computer Science, Engineering, IT, Data Science, or equivalent experience.
6+ years experience in hands-on technical roles writing production code, solution engineering, or technical architecture in large scale company / application / product environment.
7+ years of relevant Product Management experience in an agile software product development environment, with at least 3 years focused on AI/ML, conversational or agentic AI, or evaluation frameworks for production systems.
AI/ML Quality & Evaluation - Hands-on experience designing evaluation frameworks for LLMs and conversational AI: golden datasets, evaluation harnesses, human-in-the-loop validation, prompt regression testing, and red-teaming protocols.
Agentic AI Testing - Experience testing AI agents that plan and take multi-step actions: task completion and tool-call accuracy metrics, trajectory review, simulated-user and scenario-based testing, and failure analysis for handoffs, escalations, and guardrail breaches.
LLM-as-Judge Evaluation - Experience designing and calibrating LLM-as-judge systems: rubric and judge-prompt design, agreement with human raters, bias and drift controls, and use of judge scores as automated release gates.
Carrier-Grade Production Standards - Understanding of what production quality means in regulated, high-volume environments. Familiarity with SLAs, incident management, on-call rotations, and production change governance.
Business Operations - Demonstrates strong command of business-side skills (communication, customer research, product vision, feature definition), as well as technical architecture, Dev, and execution skills.
Business Analysis - Strong analytical skills with demonstrated ability to identify, analyze, and synthesize product use data and use the data to drive decisions.
Customer Experience Management - Advanced understanding of customer experience, with hands-on experience operating beta programs, reference customer pipelines, and feedback-to-product workflows for technical products at scale.
Technology - Deep understanding of platform technologies and components such as security, performance, optimization, and API integration. Advanced knowledge of full technology stack on which your assigned product runs.
Agile Methodologies - Proven success in directing matrixed resources and delivering software across multiple teams, using Agile Scrum methodologies and Agile backlog and project management tools.
Partnership Development - Experience working with external partners, vendors, and ecosystem players, including model providers and engineering partners.
Innovation Methodology - Experience applying design thinking, lean startup methodologies, rapid prototyping, and hypothesis-driven development.
Collaboration - Experience with successive elaboration and ability to develop Initiatives, Features, and User Stories that DevOps teams can ingest.
Product Management - Experience in delivering large and complex business/technology initiatives as Product Manager or lead technology role.
Communication - Proven ability to effectively and efficiently communicate with Leadership, technical, and non-technical audiences while employing a high degree of collaboration and influence. Strong requirements elicitation and technical writing skills, with the ability to present complex technical information clearly and concisely to executives and non-technical leaders.
Integration - Knowledge and experience with integration patterns, APIs, and protocols such as REST, EDI, SOAP, etc.
AI/ML Product Expertise - Strong understanding of AI technologies, capabilities, and limitations. Experience with LLMs, conversational and agentic AI, and ML platforms.
Data-Driven Decision Making - Proficiency with analytics tools and metrics-based product optimization, including fluency with SQL and Tableau or equivalent analytical solutions.
DevOps - Extensive knowledge of DevOps (CI / CT / CD) and application of CI/CT/CD in an Agile environment, including automated evaluation gates in release pipelines.

Nice To Haves

Multi-Model & Multi-Vendor Quality - Experience benchmarking and managing quality across multiple model providers and vendors objectively, not anecdotally. Comfortable defining and defending the quality scorecard that informs vendor selection and renewal.
AI Safety & Responsible AI - Working knowledge of safety guardrails, red-teaming, fairness and bias evaluation, and responsible AI practices for production-deployed AI systems.
Certified Scrum Master (CSM) - Scrum Product Owner Certification (CSPO)
Agile Certified Practitioner (PMI-ACP)

Responsibilities

Owns strategy, roadmap, and lifecycle for one or more AI products or reusable AI capabilities within an assigned product area, aligned to the broader AI portfolio strategy.
Accountable for measurable outcomes for assigned products, including adoption, retention, revenue contribution, automation impact, production quality, and customer experience improvement.
Owns the product lifecycle for complex technical solutions, ensuring alignment with business objectives and customer needs.
Defines success metrics, including quality and evaluation metrics, and tracks progress against growth and quality targets.
Conducts analysis of quantitative and qualitative data, including evaluation results and production telemetry, to identify product innovation opportunities or root cause of issues, and assess opportunity size and impact. Works with data scientists to answer complex questions or identify meaningful insights from data.
Leverages rapid hypothesis-driven testing methodologies and experiments (i.e. paper prototype, A/B testing, offline evaluation, etc.) to inform direction and prioritize investment.
Works with stakeholders and follows enterprise process to secure and maintain product funding.
Maintains current understanding of AI and technology trends, including agentic systems, evaluation methods, and competitive moves, and assesses how they impact the roadmap or create opportunity for innovation.
Owns product feature set or technical improvements to improve customer experience.
Leads customer discovery and validation activities for new AI features and products.
Writes detailed product requirements, user stories, and acceptance criteria for AI-powered features, including measurable quality and evaluation criteria.
Designs and executes rapid prototyping and MVP development cycles.
Conducts user testing and iteration based on customer feedback.
Owns launch readiness validation, confirming a product meets quality, performance, and customer experience criteria before General Availability, and runs cross-functional readiness reviews with Care, Retail, Marketing, and Engineering.
Ensures AI products deliver measurable customer value and exceptional user experience.
Advocates for customer needs throughout the development process.
Leverages customer insights for product vision, strategy, roadmap, and priorities.
Creates, manages, and fosters an active VOC feed for self and team.
Actively looks for opportunities to delight customers or meet unmet needs.
Tests ideas with real customers to ensure that the product delivers the desired benefit.
Collaborates with Technical Architects to ensure AI solutions are technically feasible and scalable.
Works with engineering teams (both internal and matrixed) to deliver products on time and within scope.
Coordinates with GTM and commercialization teams to develop launch strategies.
Drives resolution of technical and business challenges that arise during development.
Translates product strategy by writing detailed features and user stories consumable by Dev teams for high-complexity products with multiple transactions and touchpoints across teams. This work may include creation of prototypes.
Partners with PM and Dev leaders to organize effective Agile delivery teams within the Agile Release Train and Agile Teams.
Scopes and ensures alignment on the prioritization of activities based on business and customer impact.
Brings strong product and technical fluency in applied AI, including LLM-based and agentic experiences, model selection, retrieval-augmented generation (RAG), tool and API orchestration, experimentation, and production AI constraints such as latency, cost, and safety, with deep expertise in evaluation frameworks.
Collaborates with Architecture and Dev teams to ensure technical debt and long-term technical investment are factored into the roadmap.
Ensures existing production defects are factored into regular backlog prioritization for resolution based on priority.
In scaled teams, holds regular meetings and coordination activities with other PMs and Product Owners (if applicable) to ensure parallel work is in sync and dependencies are known.
Supports sales, marketing, and other stakeholder teams with product or technical knowledge and additional documentation.
Anticipates and communicates technical challenges to stakeholders and makes educated trade-off decisions with the team.
Accountable for product quality and performance in the production environment. Accountable for product and Dev team response in the event of a critical or high-impact defect, including communications to stakeholders at all levels.
Drives root cause analysis and resolution for production incidents, including AI behavior issues escalated through Care and operations teams, in collaboration with platform and product engineering.
Supports core agile practices and tenets: efficient just-in-time flow, lean practices, elimination of waste, and DevOps CI/CD.
Builds scenario libraries and simulated-user test suites covering varied personas, languages, noisy conditions, interruptions, and adversarial inputs, exercises agents end to end before release, and replays sampled production interactions to detect drift.
Designs LLM-as-judge evaluation: writes judge rubrics and prompts, calibrates judge scores against human ratings, monitors judge agreement and drift, and controls for known judge biases before judge scores are used as release gates.
Runs structured human-in-the-loop validation that reviews model and agent output for accuracy, conversational behavior, accessibility, and safety, and uses the results to calibrate automated judges and expand golden datasets.
Defines production quality standards for carrier-grade AI, including accuracy thresholds, latency budgets, task success rates, safety guardrails, and customer experience baselines that products must meet to ship and remain in production.
Builds and maintains evaluation pipelines that span multiple model providers, enabling objective model comparisons and a quality scorecard that informs model selection, retention, and renewal decisions.
Owns the quality feedback loop with model providers and engineering partners, including structured benchmarks, defect reports, and improvement requests grounded in production data.
Monitors production quality for AI-specific regressions through alerting, dashboards, and on-call protocols, in partnership with platform and product engineering.
Builds strong relationships with Network, IT, Business, Marketing, Care, and other organizations.
Negotiates priorities and resources with stakeholder groups within the product area.
Communicates product roadmaps, launches, and quality results to internal audiences and partners.
Partners with Legal, Privacy, Compliance, and Care to embed responsible AI and quality requirements at every stage of the product lifecycle.
Mentors junior product managers and quality engineers on AI evaluation methods and customer success practices.
Collaborates and develops positive working relationships with many technical and non-technical teams, including sales, commercial accounting, marketing, legal, go-to-market, finance, Dev, Architecture, and Engineering. Works with outside partners and other third parties.
Maintains professional knowledge by attending educational workshops, reviewing professional publications, and participating in professional societies.