The Senior Data Scientist owns the diagnostic and evaluation loop for LLM-based clinical services. The role investigates failure modes surfaced by clinical reviewers and product managers, develops root-cause hypotheses across prompts, context assembly, retrieval, guideline encoding, model behavior, and upstream data, and designs experiments to isolate causes. It tests candidate fixes and verifies improvements through structured evaluations while checking for regressions. The role owns the evaluation roadmap and quality bar for the team's AI services, including deployment-gating metrics, the growth of golden sets and regression suites, and evidence for determining when a service is ready to ship. It extends the existing evaluation platform rather than rebuilding it, and partners with medical directors and product managers to turn review findings into labeled evidence and prioritized evaluation criteria. The position also mentors others and may take on direct reports as the team expands. It supports the annual clinical-guidelines update cycle through regression evaluation as guidelines, prompts, and models change, ramping with the team's senior data scientists in Q4 2026. The role documents a diagnostic playbook that others can use and handles clinical data, including PHI, in accordance with organizational security, privacy, and compliance requirements.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
Job Type
Full-time
Career Level
Senior