Director, AI Experimentation & Measurement

Pfizer

1d • $162,900 - $271,500 • Hybrid

About The Position

Pfizer US Commercial is scaling a portfolio of AI initiatives, each launched as an experiment measured against a real-world baseline. The value of that portfolio depends on one thing: a shared, credible definition of what good looks like plus the evidence to prove we are reaching it. We are hiring a Director of Experimentation & Measurement to own that definition end to end. This is not a scorekeeping role. You will decide what is worth proving next, design the experiment that proves it, set the bar for success before the work begins, and tell the story of what the evidence means to the people making the biggest decisions. You are equal parts systems thinker, experiment designer, and storyteller, with the rigor to make sure the story is true. This is a build-from-scratch role for someone who can see across a whole portfolio, frame the questions that matter most, and hold the independence to say when a result does not clear the bar.

Requirements

• Bachelor's degree required with 8+ years of relevant experience spanning experimentation, measurement science, causal inference, or a quantitatively rigorous discipline — paired with strategy or product-facing work.
• Systems thinking: sees across an entire portfolio, frames the questions that matter most, and designs how proof compounds over time rather than optimizing one test at a time.
• Storytelling and executive presence: turns a rigorous, technical result into a clear, persuasive narrative that shapes senior decisions.
• Experiment design and causal credibility: has designed and defended controlled experiments at enterprise scale: power analysis, matched controls, quasi-experimental methods, pre-registration.
• Independence and spine: able to tell a senior stakeholder that a result does not clear the bar, and make it stick.
• AI fluency: evaluates AI/ML and generative systems credibly: what a real performance signal looks like versus a plausible artifact.

Nice To Haves

• Regulated-industry measurement experience.
• Provenance, auditability, and responsible-AI evaluation frameworks.
• Experience standing up an experimentation or evidence function from scratch.
• Breadth of diverse leadership experiences and capabilities including: the ability to influence and collaborate with peers, develop and coach others, oversee and guide the work of other colleagues to achieve meaningful outcomes and create business impact.

Responsibilities

• Own the portfolio-wide standard for what counts as proof—defined before work begins, not after results land. Draft and socialize pre-registered success criteria for each initiative (primary KPIs, guardrails, minimum detectable effect, read windows). Maintain a shared measurement playbook (matched controls, power calculations, blinded review, backtesting where appropriate).
• Prioritize what the portfolio must prove next—and in what sequence evidence compounds. Maintain a ranked learning agenda across initiatives and recommend the next proof point per initiative.
• Review experiment designs: control arms, audience matching, read windows, confounders, data leakage risks. Run power analysis and flag underpowered or ungradeable designs before launch.
• Produce evidence read memos on a fixed cadence: what was tested, what was observed, what it means, what is recommended. Synthesize across initiatives into one portfolio story—not a patchwork of disconnected scorecards.
• Guard against failure modes that quietly invalidate results: moving goalposts; letting the people a system is meant to outperform grade it; acting before pre-registered proof exists. Escalate when a read is not gradeable.
• Run evidence read sessions at pre-registered dates; own continue / adjust / stop recommendations with rationale; track whether recommendations are acted on.

Benefits

• 401(k) plan with Pfizer Matching Contributions and an additional Pfizer Retirement Savings Contribution
• Paid vacation, holiday and personal days
• Paid caregiver/parental and medical leave
• Health benefits to include medical, prescription drug, dental and vision coverage
• Relocation assistance may be available

What This Job Offers

• Job Type: Full-time
• Career Level: Director