Product Manager, Evaluation

Learning Commons • Redwood City, CA

2h • Hybrid

About The Position

Learning Commons is Mark Zuckerberg and Priscilla Chan’s education initiative, which aims to scale proven teaching and learning practices to benefit every learner. Learning Commons became the name of our education efforts in 2025 to build on the Chan Zuckerberg Initiative’s work over the past decade to advance learning science and help translate that research into classroom practice.
The Team
At Learning Commons, we pair technology with grantmaking to scale proven teaching and learning practices to benefit every learner. We aim to translate what learning science tells us about how students learn best into classroom practice. With the advent of generative AI, that translation work can be accelerated and scaled to have a greater impact.
Our mission is to bring learning science into the tools used every day by teachers and students, ensuring that technology reflects the realities of classrooms and strengthens teaching and learning.
In today’s fragmented edtech landscape, school districts are often left piecing together products that don’t always align with curricula or instructional needs. While AI holds enormous potential to support educators, it can only deliver on that promise when grounded in research, high-quality educational data, and expert evaluation. That’s why we’re building open, public-purpose infrastructure — datasets, rubrics, and resources — that help raise the standard for educational tools and create more consistent, impactful learning experiences for all students and teachers.
At our core, we are builders, and our unique builder philanthropy approach is what sets us apart from other education funders. Take a closer look at the highlights and significant milestones of CZI’s first eight years of education work.
The Opportunity
Educators are already using AI-based tools in various ways—including generating lesson plans, creating classroom materials such as tests and assignments, and helping differentiate instruction for students. At Learning Commons, we are inspired and excited about the possibilities and promise of AI to accelerate the availability of research-backed practices at scale. We are actively and thoughtfully exploring how to incorporate AI into products in close partnership with researchers, experts, and educators.

As a Product Manager on the Evaluators team, you'll own the dataset annotation and evaluator development pipeline that helps EdTech developers build better products. You'll partner with data scientists, engineers, learning scientists, and researchers to create high-quality evaluators grounded in learning science. Starting with literacy and expanding into student feedback, math, science, and beyond, you'll be doing frontier work: bringing pedagogical expertise into AI evaluation in ways that haven't been done before. This is an opportunity to have hands-on impact at the intersection of AI/ML, learning science, and product development.

Requirements

• 3-5 years of product management experience: A track record of shipping products, with direct experience working on AI/ML products, data annotation pipelines, or model evaluation workflows
• Strong technical foundation in AI/ML - Comfort discussing annotation methodologies, model performance metrics, accuracy thresholds, inter-rater reliability, and evaluation frameworks with data scientists; ability to understand technical tradeoffs and make informed product decisions
• Exceptional communication and storytelling skills - Ability to translate complex technical and pedagogical concepts into clear narratives for different audiences; skilled at building alignment across diverse stakeholders and making compelling cases for prioritization decisions
• Self-directed execution mindset - Thrives in ambiguous, frontier environments where playbooks don't exist; takes initiative to identify problems, propose solutions, and drive outcomes; comfortable making decisions with incomplete information and learning through iteration
• Customer-centric approach - Passion for understanding developer needs through direct engagement and feedback loops; commitment to building products that solve real problems; driven by mission-oriented work in education

Responsibilities

• Own the annotation-to-evaluator pipeline: Manage the end-to-end process from identifying pedagogical constructs through dataset creation, model training, and evaluator validation; balance accuracy requirements with delivery timelines
• Shape evaluator roadmap and prioritization: Conduct discovery with EdTech developers building literacy and content generation tools to understand which evaluators deliver the most value; provide input on prioritization based on impact, feasibility, and strategic alignment
• Make strategic tradeoffs between accuracy and velocity: Partner with data scientists to set appropriate accuracy thresholds for different evaluator types; decide when to ship, iterate, or restart based on performance against baselines
• Translate learning science into product specifications: Work with learning scientists to convert pedagogical frameworks for literacy instruction into clear, measurable evaluation criteria; write detailed specs that guide annotation and model development
• Optimize and scale the annotation workflow: Identify opportunities to improve annotator efficiency, reduce ambiguity in guidelines, and increase inter-rater reliability; document learnings to enable expansion into new subject areas

Benefits

• Provides a generous employer match on employee 401(k) contributions to support planning for the future.
• Paid time off to volunteer at an organization of your choice.
• Funding for select family-forming benefits.
• Relocation support for employees who need assistance moving

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Education Level: No Education Listed