Staff Data Scientist - Experimentation & Causal Inference

HighLevel

2d•Remote

About The Position

HighLevel is seeking its first Staff Data Scientist, Experimentation & Causal Inference to establish how the company learns from experiments and translates these findings into reliable product decisions and business strategies. While teams are already conducting experiments for data-driven decision-making, this role will introduce the necessary rigor and consistency to scale this practice across the organization. The successful candidate will define company-wide standards for experiment design and causal inference, integrate these practices into product development, and mentor Product Managers and analysts in running robust tests. This position operates within a dynamic, multi-product SaaS/CRM environment characterized by small sample sizes, concurrent product development, and the high cost of incorrect "win" assessments. This is a foundational, hands-on individual contributor role with executive backing, offering a pathway to build a Data Science team as the function matures.

Requirements

9+ years of experience in data science, product analytics, or applied statistics, with extensive hands-on experience in designing and analyzing large-scale online controlled experiments.
Strong foundation in applied statistics, including frequentist principles, Bayesian methods, power analysis, variance reduction, and understanding the failure modes of A/B testing (e.g., peeking, multiple testing, network/cluster effects).
Practical experience in causal inference with sound judgment regarding the distinction between causal results and data generation artifacts.
Experience in small-sample, fast-paced, multi-product environments, with the ability to discern when a clean experiment is necessary versus when a rapid, sufficiently accurate analysis is appropriate.
Proficiency in SQL and working knowledge of Python or R.
Proven ability to influence cross-functional teams and senior leadership, improving experiment quality without direct authority.

Nice To Haves

Familiarity with modern experimentation platforms like Statsig.
Experience in establishing an experimentation practice or culture from the ground up.
Background in B2B SaaS, CRM, or product-led growth, with an understanding of the measurement challenges inherent in these models.
Experience with multi-tenant or marketplace products, particularly those with nested structures like agency -> sub-account -> end-customer.

Responsibilities

Define the end-to-end methodology for all teams, covering hypothesis, metrics, design, power, readout, and decision-making, and establish it as the default process.
Own the statistical approach for small-sample, fast-paced contexts where traditional A/B testing power is challenging to achieve, including significance, multiple comparisons, sequential testing, and variance reduction techniques like CUPED.
Develop methods for analyzing clustered, hierarchical data (user -> sub-account/location -> agency), accommodating varying randomization and analysis units.
Apply rigorous causal inference methods (e.g., matching, diff-in-diff, instrumental variables, synthetic control) for situations where clean experiments are not feasible, such as churn, onboarding, and go-to-market strategies, to distinguish genuine signals from selection bias, seasonality, and mix effects.
Oversee the design discipline for managing multiple concurrent experiments, including layering, orthogonal experiments, holdouts, and guardrails to prevent interference between tests.
Collaborate with AI/ML teams to design and evaluate experiments for AI features, including measurement for non-deterministic and rapidly iterating systems.
Lead the experiment review forum, ensuring the validity of results.
Develop an Experimentation curriculum and templates to enhance the skills of PMs and analysts, enabling the scaling of good design practices beyond the individual.
Partner with Analytics Engineering to ensure governed, experiment-ready data and consistent metric definitions.
Influence leadership and cross-functional partners by translating statistical nuances into clear, actionable guidance for investment decisions.