Strategy & Analytics Experimentation Director

Faire•San Francisco, CA

2h•$276,000 - $379,500•Hybrid

About The Position

We are looking for an Experimentation Director to be Faire's authority on experimentation. Your mission is to increase the rigor of experimentation at Faire without sacrificing velocity. You'll own our experimentation methodology and principles, and raise the quality of how experiments are designed, run, read, and acted on across every team. This is a high-visibility, high-impact individual contributor role reporting into the Head of Strategy & Analytics. You will be the default experimentation partner and reviewer for our most high-stakes and complex experiments, and a key partner to Strategy & Analytics, Engineering, Data Science, and Product on everything experimentation-related.

Requirements

10+ years of relevant experience in experimentation, data science, or a closely related analytical role.
Extensive experimentation experience. You have designed, reviewed, and analyzed hundreds of experiments — ideally on a central experimentation or platform team at a company with a mature testing culture.
Deep statistical expertise. You have a strong command of the methods that make experiments trustworthy: power and sample sizing, sequential testing, variance reduction, multiple comparisons, and interference between units. When a clean A/B test isn't possible, you can design and run alternatives.
Analytically rigorous. You read results skeptically and catch subtle problems others miss.
Strong cross-functional influence. You can raise the bar on how teams across Engineering, Product, Data Science, and Strategy & Analytics design, run, and interpret experiments.
Understand data and experimentation platforms — how assignment infrastructure, metric pipelines, and data models work, and how methodological choices interact with them.
Strong communicator. You make our experiments and their impact legible to others at Faire through strong spoken and written communication.
Practical. You balance rigor with speed and optimize for impact.

Responsibilities

Own experimentation methodology and principles to ensure accuracy and rigor across all experiments.
Build and maintain Faire's experimentation OS, including experiment review processes and cross-company experiment tracking and visibility.
Adapt our experiment methodology and operating system as velocity of experimentation increases substantially with AI.
Partner with S&A, Engineering, Data Science, and Product to design and measure experiments and act as the default design partner and reviewer for high-stakes or methodologically complex experiments.
Act as Faire's expert on causal inference in cases where we cannot run a true experiment, including helping other teams design and execute diff-in-diff, synthetic control, and other causal analyses.
Perform ad hoc experiment review and analysis to understand anomalous results and drive each to a trusted conclusion.
Partner with the Analytics Platform team to help set their roadmap for the experiment platform and data model.