Research Methods Lead

StudyFetch•Beverly Hills, CA

5d•Onsite

About The Position

We are a technology company building AI-native learning products used by more than seven million students worldwide, alongside Honen, our workforce-learning platform for organizations. Both run on the Learn Engine, the intelligence that moves a learner from initial understanding to demonstrated mastery. We work with partners like NVIDIA to bring responsible, learning-first AI to the students who need it most. This role is crucial for building the proof and evidence base that would satisfy external reviewers, such as district procurement reviewers, academic peer reviewers, and funders. You will design studies, own instruments, and build the evidence base to support effectiveness claims for StudyFetch and Honen that can be defended to anyone. This is a founding-team role with direct collaboration with decision-makers, and you will set the standard for all future studies.

Requirements

PhD in Learning Sciences, Educational Psychology, Psychometrics, Applied Statistics, or a related quantitative field, plus 7+ years applying it to real research programs, OR fewer credentials with a track record of building evidence that held up to outside scrutiny (ESSA/WWC-aligned or equivalent, published, reviewed, defended).
Experience building evidence for a real audience, not just internal reports.
Ability to articulate a study or benchmark design, external reviewer challenges, and lessons learned.
Strong understanding of causality, experimental and quasi-experimental design, sample and power analysis, and the distinction between proven and estimated results.
Clear and effective written and verbal communication skills, adaptable to various audiences (engineers, founders, academic partners).
Proficiency in stating uncertainty quantitatively when possible and qualitatively when not.
Experience navigating IRB and human-subjects research, treating it as an integral part of the design.
A strong conviction in the mission of improving learning for all individuals.
Research design: experimental and quasi-experimental methods, pre-registration, sample and power analysis, causal inference.
Measurement: psychometrics, item response theory, rubric and codebook development, interrater reliability (human and LLM-assisted raters).
Evidence standards: ESSA/WWC tiers or equivalent, IRB protocols, human-subjects research, data-sharing agreements.
Analysis: Python (Pandas, NumPy, SciPy) or R, expert-level SQL, hypothesis testing at scale.
AI/LLM familiarity: eval frameworks, LLM-as-judge and its failure modes, evaluating AI-assisted rating pipelines.
Reporting: technical reports and evidence packages that can withstand external review.

Nice To Haves

Prior experience with children's data and associated governance rules.
Experience publishing or presenting research externally.

Responsibilities

Define the evidence strategy, including deciding which effectiveness claims to prove, the standard of proof (e.g., ESSA/WWC-aligned for education, equivalent for workforce training), and assessing claim defensibility.
Design outcome measures and study designs (quasi-experimental, randomized) to overcome the limitations of internal scoring and prove platform effectiveness.
Build an externally acceptable benchmark, including the rubric, codebook, and interrater reliability process for human and AI raters.
Enhance external credibility by recruiting and managing an independent advisory panel and methods reviewers, leading peer-review and publication efforts, and representing research to academic partners, procurement evaluators, and the press.
Develop a research playbook, reusable instruments, and operating patterns to make future studies more efficient and cost-effective.
Manage ethics and governance across different populations (K-12, higher-ed, enterprise learners), including IRB protocols, human-subjects determinations, and data-sharing agreements.
Collaborate closely with product and go-to-market teams to identify evidence requirements early, preventing launch blockers related to claim defensibility.