Principal Product Manager
About The Position
Copilot's usefulness lives or dies on whether it actually does what people need reliably, safely, and across an enormous range of real enterprise and consumer scenarios. We're looking for a Principal Product Manager to own how we measure that. You'll build and run the evaluation systems that determine whether a model update, a new Copilot skill, or an agentic workflow is ready to ship. You'll partner directly with applied research, model training, and engineering teams to turn "does this feel better" into concrete, defensible metrics and you'll use those metrics to shape what we prioritize next. This is a technical, judgment-heavy PM role. You won't just write requirements docs; you'll personally dig into transcripts, build eval sets, argue about what "good" means for a given surface, and make the call on whether something is ready for customers. Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.
Requirements
- Bachelor's Degree AND 8+ years experience in product/service/program management or software development OR equivalent experience.
Nice To Haves
- Bachelor's Degree AND 12+ years experience in product/service/program management or software development OR equivalent experience.
- 4+ years experience taking a product, feature, or experience to market (e.g., design, addressing product market fit, and launch, internal tool/framework).
- 6+ years experience improving product metrics for a product, feature, or experience in a market (e.g., growing customer base, expanding customer usage, avoiding customer churn).
- 6+ years experience disrupting a market for a product, feature, or experience (e.g., competitive disruption, taking the place of an established competing product).
Responsibilities
- Own the eval lifecycle end-to-end: design, build, and maintain evaluation suites (offline benchmarks, live A/B experiments, agentic task suites, red-team/adversarial sets) that measure quality, safety, and task success across Copilot surfaces (Microsoft 365, Windows, Edge, Copilot Studio agents, etc.)
- Gate model and feature launches: define entry/exit criteria for shipping model updates or new capabilities, and make the ship/no-ship call in partnership with research and engineering leadership
- Translate model behavior into product requirements: read outputs and failure modes directly, identify patterns, and turn them into prioritized fixes — whether that's a training data gap, a prompt/system-message change, or a UX guardrail
- Build feedback loops: stand up pipelines that turn user feedback, support signals, and enterprise customer escalations into structured eval cases, so regressions get caught before they reach customers
- Partner cross-functionally: work daily with applied scientists, ML engineers, Responsible AI, and Trust & Safety teams to represent user needs in model development, and represent model capabilities/limitations back to the broader product org
- Define what "good" looks like for ambiguous, high-stakes scenarios (enterprise data handling, agentic actions taken on a user's behalf, multi-turn reasoning) where there's no existing playbook
- Communicate quality tradeoffs clearly to leadership — what shipped, what didn't, what the residual risk is, and why.
Benefits
- Certain roles may be eligible for benefits and other compensation.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Principal
