Physician Reviewer (MD/DO) - Medical AI Evaluation

Mindrift

1d • $70

About The Position

Mindrift connects specialists with project-based AI opportunities for leading technology companies, focused on testing, evaluating, and improving AI systems. This opportunity is for US-based, actively practicing physicians (MD/DO) contributing to a medical AI evaluation program, primarily focused on general and internal medicine. Physicians evaluate clinical EHR vignettes and benchmark items across multiple body systems, assess clinical reasoning and answer quality, and provide short rationales. Work involves three independent blind reads followed by physician adjudication. Participation is project-based, with an estimated 10–20 hours per week during active project phases; workload is not guaranteed and tasks must meet acceptance criteria and deadlines. Compensation is paid per accepted task, with rates depending on qualification tier and task efficiency, up to the equivalent of $70 per hour.

Requirements

• Medical degree (MD or DO).
• Active, unrestricted US medical license, verified with the issuing medical board.
• Board certification or completed residency training, with recent direct patient care; attending-level experience is preferred.
• Broad, multi-system diagnostic and treatment experience in internal, family, emergency, or hospital medicine.
• Strong clinical reasoning, including differential diagnosis, next-step management, and application of evidence-based guidelines.
• Strong written English proficiency (C1+).
• Must be a US-based, actively practicing physician; the opportunity is not suitable for medical students, unlicensed medical graduates, non-physician clinicians, or non-clinical healthcare titles.
• Submit a CV in English and indicate English proficiency.

Nice To Haves

• Prior experience in medical AI evaluation, clinical content or exam-question review, or medical annotation/quality assurance.
• Attending-level experience.

Responsibilities

• Evaluate clinical EHR vignettes paired with questions, proposed answers, and distractor trap answers across diagnosis and treatment tasks and multiple body systems.
• Score benchmark items for vignette accuracy and completeness, whether the vignette gives away the answer, answer correctness and gradeability, and trap quality.
• Check whether the reasoning chain reaches the answer using vignette facts alone, assess reasoning traces, and write short rationales.
• Participate in three independent blind reads followed by physician adjudication.
• Submit tasks by the deadlines and meet the stated acceptance criteria.

What This Job Offers

• Job Type: Part-time
• Career Level: Senior
• Education Level: Ph.D. or professional degree