Clinical AI Quality Specialist

Hello Heart
•$130,000 - $143,000

About The Position

Hello Heart is an AI company focused on heart health. Its platform aims to predict and prevent cardiac events, and its AI team builds Nia, an AI health assistant for people managing cardiovascular health. The Clinical AI Quality Specialist evaluates and monitors Nia’s clinical behavior before and after release, working alongside the Clinical AI Quality & Governance Team Lead. This hands-on role involves running evaluations, reviewing production conversations, and turning findings into documented decisions with product and engineering. The specialist maintains clinical specifications, review records, and evidence so they are current and auditable, and writes precise clinical assessments that enable product and engineering teams to act.

Requirements

  • PharmD or equivalent clinical doctorate.
  • Active U.S. clinical license.
  • At least 3 years of combined clinical practice and clinical AI evaluation.
  • Experience designing or running evaluations for LLM-based systems, reviewing production conversations, and using monitoring tools such as Langfuse.
  • Command of triage and escalation, crisis and self-harm handling, and medication safety, grounded in current guidelines.
  • Ability to investigate assumptions, consult current guidelines, drug labels, and primary sources, and cite sources in written assessments.
  • Ability to use AI tools to work efficiently and verify their output against sources before relying on it.
  • Excellent written and verbal communication, including the ability to explain clinical risk to engineering and product audiences and write precise clinical assessments and documentation.
  • Ability to accept feedback without defensiveness.
  • Curiosity, openness to learning, and comfort with ambiguity in a fast-moving field.
  • Genuine interest in how AI can improve preventive care and chronic disease management.

Nice To Haves

  • Clinical experience in cardiovascular care or hypertension management.
  • Experience with structured rating of clinical transcripts, including measuring agreement between raters.
  • Experience reviewing crisis and self-harm behavior.
  • Experience designing or running evaluations for LLM-based systems, or using monitoring tools such as Langfuse.

Responsibilities

  • Run offline evaluations of clinically relevant changes to Nia.
  • Write test cases, including tricky and ambiguous cases, and score outputs against expected clinical behavior.
  • Build and maintain evaluation datasets and contribute to scoring methods.
  • Review production conversations using defined sampling methods; participate in weekly reviews and help prepare monthly results that state what was and was not covered.
  • Test Nia’s detection and handling of high-risk situations, including medical emergencies, self-harm, and medication safety, and measure over- and under-escalation.
  • Describe failures clearly for engineering, identify the likely cause where possible, and ensure each unsafe or incorrect finding has an owner, ticket, and decision.
  • Keep clinical specifications, review records, and evidence current and auditable.
  • Write clear, precise clinical assessments and documentation for engineering and product.
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service