Principal Product Manager- Fine-Tuning & Distillation (CoreAI)

Microsoft • Boston, MA

16h

About The Position

The Principal Product Manager for Fine-Tuning and Distillation in Microsoft’s CoreAI organization is a senior product leader responsible for shaping the strategy, execution, and impact of advanced model customization capabilities within Microsoft Foundry. This role sits at the center of Microsoft’s AI platform strategy, partnering closely with flagship internal customers including M365, GitHub, Microsoft AI to turn internal innovation and proof points into scalable, secure, and world-class platform capabilities for external developers and enterprises. By driving the end-to-end product vision for fine-tuning and distillation, this role is critical to enabling differentiated model performance, cost efficiency, and customization at scale, while helping establish Microsoft Foundry as the industry’s leading AI platform.

At Microsoft, our mission is to empower every developer and organization to achieve more with AI. Within the CoreAI organization, Microsoft Foundry is the platform where cutting-edge AI research, infrastructure, and product innovation converge to power Microsoft’s most important AI experiences—and to bring those same capabilities to customers worldwide.
As the Principal Product Manager for Fine-Tuning and Distillation, you will define and drive the vision for model customization capabilities across Microsoft Foundry, enabling teams to adapt frontier and domain-specific models for performance, cost, latency, and safety. This role sits at the intersection of platform, research, and applied AI, working closely with Microsoft’s largest internal customers such as M365, GitHub, Microsoft AI, and other flagship AI teams to turn internal innovation and proof points into scalable, secure, and developer-friendly platform offerings for external customers.
You will partner deeply with engineering, research, applied science, and partner teams to design end-to-end fine-tuning and distillation workflows that support a range of scenarios, from enterprise customization to high-volume consumer workloads.

This role is critical to establishing Microsoft Foundry as a world-class AI platform by making advanced model optimization techniques accessible, reliable, and cost-effective at scale.
We are looking for an AI/ML product leader who combines technical intuition with platform thinking - someone who thrives in ambiguity, can translate frontier research into practical systems, and has a track record of shipping foundational capabilities used by multiple teams at scale.

Requirements

• Bachelor's Degree AND 8+ years experience in product/service/program management or software development OR equivalent experience.
• Ability to meet Microsoft, customer and/or government security screening requirements are required for this role.
• Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter.

Nice To Haves

• 3+ years of hands-on experience with Azure (or similar cloud) services, including resource provisioning, IAM, networking, and scripting (PowerShell or Azure CLI).
• 6+ years experience managing cross-functional and/or cross-team projects.
• Deep expertise in AI/ML data pipelines—pre-training, fine-tuning, evaluation, and deployment.
• Track record of leading cross-functional efforts at scale in high-stakes, fast-paced environments.
• Startup founder/early-stage experience or leadership roles at Meta, Google, Amazon, or similar AI-first companies
• Proven experience in product management for large-scale data or AI platforms.
• Technical background in data lifecycle management, security, and compliance.
• Demonstrated ability to lead cross-functional teams and drive strategic initiatives.
• Entrepreneurial mindset; thrives in fast-paced, ambiguous environments.

Responsibilities

• Platform Vision & Strategy: Define and drive the long-term vision and roadmap for fine-tuning and distillation capabilities within Microsoft Foundry.
• Customer-Informed Design: Partner closely with flagship internal teams (M365, GitHub, Microsoft AI, and others) to identify needs, validate solutions, and convert internal innovation into scalable platform features.
• End-to-End Workflows: Own the product experience for fine-tuning and distillation, spanning data preparation, training, evaluation, deployment, and lifecycle management.
• Model Optimization: Enable techniques that improve model performance, cost efficiency, latency, and safety across diverse workloads and model families.
• Research-to-Product Translation: Collaborate with research and applied science teams to operationalize cutting-edge techniques into reliable, supportable platform capabilities.
• Developer Experience: Design intuitive APIs, tooling, and abstractions that make advanced model customization accessible to a broad developer audience.
• Cross-Functional Leadership: Drive alignment across engineering, infrastructure, security, compliance, and business stakeholders to deliver high-impact outcomes.
• Roadmap & Prioritization: Establish a customer-driven prioritization framework balancing near-term impact with long-term platform scalability.
• Operational Excellence: Define success metrics, feedback loops, and continuous improvement processes to ensure reliability and adoption at scale.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Number of Employees: 5,001-10,000 employees