Senior Data Scientist- Experimentation & Causal Inference

WorkInCryptoGlobal • San Francisco, CA

4d • Onsite

About The Position

We are seeking a Senior Data Scientist – Experimentation & Causal Inference to serve as the statistical authority behind our rapidly expanding experimentation platform. This role is responsible for ensuring that every experiment we run is trustworthy, sensitive, and correctly interpreted. You will own the methodology layer of experimentation—not the infrastructure. While our platform engineering team builds and operates the experimentation platform, you will design, validate, and continuously improve the statistical frameworks that power it. You will be the expert the organization relies on when asking: "Can we trust this result?" More importantly, you will build the methodologies and automated validation frameworks that answer that question before it needs to be asked. You will join a multidisciplinary team of data scientists and platform engineers within the Big Data organization, reporting directly to the Head of Big Data. The experimentation platform engineering team will implement your statistical specifications into production systems, allowing you to focus on methodology, experimentation quality, and causal inference.

Requirements

• Master's or PhD in Statistics, Biostatistics, Economics (Econometrics), Computer Science, Mathematics, or a related quantitative discipline.
• 3+ years of industry experience designing and analyzing online controlled experiments at scale within a technology company.
• Demonstrated experience working with large-scale digital experimentation programs involving meaningful user traffic. Experience beyond survey-based or clinical trial experimentation is strongly preferred.
• Strong foundation in hypothesis testing, power analysis, multiple testing correction, sequential testing, and experimental design.
• Hands-on experience with at least two of the following: CUPED or other variance reduction techniques, Sample Ratio Mismatch (SRM) detection, Continuous experimentation quality monitoring, Experiment health diagnostics.
• Advanced Python proficiency, including experience with SciPy, Statsmodels, NumPy, or equivalent statistical libraries.
• Strong SQL skills, including Window functions and Common Table Expressions (CTEs).
• Experience building simulations and statistical validation frameworks.
• Ability to explain complex statistical concepts to technical and non-technical stakeholders.
• Strong capability to translate business questions into rigorous experimental methodologies.
• Comfortable writing clear specifications that engineers can implement in production systems.
• Extensive hands-on experience using AI-assisted development tools such as Claude Code, OpenClaw, Cursor, or similar AI coding assistants.
• Proven ability to leverage AI tools to accelerate analysis, automate repetitive workflows, improve productivity, and build self-service tooling.
• Fluent in both English and Chinese.
• Ability to collaborate effectively with engineering, product, and data teams across APAC and North America.
• Comfortable serving as a communication bridge between the US R&D Center and regional teams.

Nice To Haves

• Experience with Bayesian experimentation frameworks.
• Familiarity with multi-armed bandit methodologies.
• Understanding of streaming data ecosystems and their impact on experimentation validity, including Kafka, Flink, and real-time analytics architectures.
• Publications or conference presentations related to experimentation, causal inference, statistical methodology, or machine learning evaluation.
• Experience in Fintech, Cryptocurrency, Trading platforms, or other high-scale consumer technology environments.

Responsibilities

• Own Experiment Quality & Guardrail Frameworks: Design and maintain automated anomaly detection systems for live experiments, including Sample Ratio Mismatch (SRM) detection, traffic allocation validation, and experiment integrity monitoring. Define alert thresholds and circuit-breaking criteria to identify or stop compromised experiments before they influence business decisions. Develop and validate guardrail metrics that balance sensitivity, directionality, and interpretability. Establish experimentation standards and best practices across the organization.
• Improve Experiment Sensitivity Through Variance Reduction: Design and implement variance reduction methodologies such as CUPED, covariate adjustment techniques, and pre-experiment normalization methods. Reduce metric noise caused by historical user behavior and external factors. Enable faster and more reliable detection of treatment effects, particularly in low-traffic environments, high-priority product launches, and time-sensitive business decisions. Drive measurable improvements in experimentation efficiency, with a target of reducing average experiment duration from 14 days to 9 days and significantly increasing experimentation throughput.
• Lead Continuous A/A Testing & Data Quality Monitoring: Design and operate continuous A/A testing programs as a permanent health check for the experimentation ecosystem. Validate the integrity of the end-to-end measurement pipeline, including client-side instrumentation, event collection systems, message queues, and real-time data warehouses. Monitor baseline metric stability, pipeline reliability, and instrumentation quality. Define what constitutes a statistically healthy experimentation environment and provide specifications for engineering teams to operationalize monitoring.
• Partner on Experiment Design & Causal Inference: Collaborate with Product, Engineering, Growth, and Leadership teams on experimental design, sample size calculations, power analysis, metric selection, duration estimation, and result interpretation. Provide guidance on causal inference methodologies when randomized experimentation is not feasible, including Difference-in-Differences (DiD), Synthetic Control, and Regression Discontinuity Design (RDD). Ensure business decisions are supported by rigorous statistical reasoning.
• Enable Scalable Self-Service Experimentation: Translate statistical methodologies into technical specifications, validation scripts, decision frameworks, and best-practice guidelines. Empower experiment owners to self-serve while maintaining scientific rigor and consistency.

Benefits

• Define Experimentation Standards at Massive Scale: You will establish the statistical foundations for experimentation across a platform serving 80M+ users globally. Your methodologies will directly influence product strategy and business decisions.
• Immediate and Measurable Impact: With more than 120 active experiments and ambitious growth plans, your work will be deployed quickly and deliver visible business outcomes.
• Science-Driven Decision Making: We run experiments to discover the right answers—not to validate predetermined decisions. You will have direct visibility with leadership when experimental findings challenge conventional thinking.
• High-Caliber, Low-Bureaucracy Environment: Work alongside a focused team of data scientists and engineers where responsibilities are clearly defined: You own the science. Engineers own the platform. Great ideas move quickly into production.
• This is a rare opportunity to shape the experimentation culture, methodology, and decision-making framework of a global-scale technology company.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior