Research Scientist, Clinical AI Evaluation Lead

Meta•Burlingame, CA

1d•$184,000 - $257,000

About The Position

Meta's products give people the power to build community and bring the world closer together. The Meta Health team supports this mission through the discovery, development, and delivery of clinically validated health products with the potential to improve both individual and public health outcomes, at Meta scale. We are seeking a Research Scientist, Health Generative AI Evaluation Lead to help build a Generative AI health program aimed at empowering people to live healthy lives throughout the world. This is a hands-on individual contributor role with organization-wide clinical and technical influence. You will set the evaluation strategy for Health GenAI, build the benchmarks and harnesses that back it, and partner with clinicians, AI engineers and scientists, product, and regulatory teams to connect model-level metrics to clinical meaning. You bring the experience and ability to deliver high-quality work in a fast-paced, broad-scope, product-minded environment. The impact you'll make: The mission of the Meta Health team is to improve people's health that includes but is not limited to social, mental, and physical health through innovative technology solutions. We strive to make a positive impact on individuals' lives by addressing various health challenges and promoting overall wellness. Our team is dedicated to developing cutting-edge experiences that cater to diverse health needs, with the ultimate goal of enhancing the quality of life for our users. By working together, we aim to create a healthier and happier society, one step at a time.

Requirements

10+ years of experience in AI/ML or a quantitative research field, including 3+ years designing and running evaluations of LLMs or foundation models.
Experience building clinical benchmarks, rubrics, and measurement methodology (safety, factuality, calibration, and bias)
3+ years applying measurement science to generative AI: constructing benchmarks and eval sets, defining scoring rubrics, validating human and model-based graders, and quantifying uncertainty in model quality claims
Experience applying machine learning or GenAI to healthcare or medical problems (particularly in electronic health records) in direct partnership with clinical domain experts, including at least one system deployed to real users via real-world clinical confirmation study
Experience setting technical direction across organizational boundaries, influencing clinical, product, engineering, and regulatory partners
Strong publication record at top-tier venues or high-impact peer-reviewed journals
MD and/or PhD in Biomedical Informatics, Machine Learning, Computer Science, Statistics, or a related field, or equivalent practical experience
Bachelor's degree in Computer Science, Computer Engineering, relevant technical field, or equivalent practical experience

Nice To Haves

Experience with clinical validation, regulated medical device software (SaMD), or FDA submissions including judgment about when a product belongs on a general-wellness path versus a regulated one
Track record of taking a system from inception through production, including defining the evaluation criteria that gated its launch
Experience evaluating models in production at consumer scale: online metrics, safety regression across base-model updates, and adversarial testing

Responsibilities

Own the evidence standard for Health GenAI launches: define the readiness bar, produce the evidence package behind each launch decision, and serve as the scientific authority that decision rests on.
Define and lead the evaluation strategy for GenAI health products, establishing what “good,” “safe,” and “ready to ship” mean, and the evidence required to claim each.
Design and build LLM and multimodal benchmarks for health: clinical reasoning, factuality and hallucination, safety and harm avoidance, appropriate escalation and refusal, calibration and uncertainty, and fairness across populations.
Develop clinical confirmation methodology, pairing automated evaluations with clinician-in-the-loop adjudication, inter-rater reliability, and study designs that produce defensible evidence.
Partner with product and AI engineering from concept through launch of Meta's Health GenAI features, defining the claim, the evidence, and the guardrails before the feature exists.
Partner with clinical research, regulatory, and legal to align evaluation evidence with submission and documentation needs where products are on a regulated path.
Define new research directions in health AI evaluation where no established methodology exists, and publish work that shapes how the field measures clinical generative AI.
Set technical direction across teams, raise the scientific bar through review and mentorship, and represent Meta's position on Health AI evaluation externally through publication, benchmarks, and standards work.
Generates entirely new research directions, works on problems of the highest complexity and ambiguity, and makes technical contributions likely to reshape how the field thinks about the area.