Licensed Physician (MD/DO) - Freelance AI Trainer

Mindrift • Lebanon, PA

5h • $100

About The Position

Mindrift connects specialists with project-based AI opportunities for leading tech companies focused on testing, evaluating, and improving AI systems. This opportunity is for US-based, actively practicing physicians (MD/DO) to evaluate clinical EHR vignettes for a medical AI evaluation program, with general and internal medicine as the main focus. Work is project-based and non-permanent.

Contributors evaluate clinical vignettes and related questions, proposed answers, and distractor answers across multiple body systems. They assess benchmark-item quality, clinical reasoning, answer correctness, and reasoning traces, and write short rationales. The workflow involves three independent blind reads followed by physician adjudication. Tasks are estimated at 10–20 hours per week during active project phases; this is not a guaranteed workload. Tasks must meet acceptance criteria and deadlines. Compensation is paid per accepted task, with rates dependent on qualification tier and task efficiency, up to the equivalent of $100 per hour.

Requirements

• Medical degree (MD or DO).
• Active, unrestricted US medical license, verified with the issuing medical board.
• Board certification or completed residency training.
• Recent direct patient care; attending-level experience is preferred.
• Broad, multi-system diagnostic and treatment experience in internal, family, emergency, or hospital medicine.
• Strong clinical reasoning, including differential diagnosis, next-step management, and application of evidence-based guidelines.
• Strong written English (C1+).
• Must be a US-based, actively practicing physician; medical students, unlicensed medical graduates, non-physician clinicians, and non-clinical healthcare titles are not eligible.

Nice To Haves

• Prior experience in medical AI evaluation, clinical content or exam-question review, or medical annotation/QA.

Responsibilities

• Evaluate clinical EHR vignettes paired with questions, proposed answers, and distractor “trap” answers across diagnosis and treatment tasks.
• Score benchmark items for vignette accuracy and completeness, whether the vignette gives away the answer, answer correctness and gradeability, and trap quality.
• Assess whether reasoning chains reach answers using only the vignette facts, evaluate reasoning traces, and write short rationales.
• Participate in three independent blind reads followed by physician adjudication.
• Submit tasks by their deadlines and meet the stated acceptance criteria.

What This Job Offers

• Job Type: Part-time
• Career Level: Mid Level
• Education Level: Ph.D. or professional degree