Physician (MD/DO) - Medical AI Evaluation

Mindrift

1d • Remote

About The Position

Mindrift is seeking actively practicing physicians with an active, unrestricted US medical license to evaluate clinical EHR vignettes for a medical AI evaluation project, with a primary focus on general and internal medicine. The project is time-limited and project-based, and is intended to fit around clinical practice. There is no patient contact; contributors review written clinical cases.

Physicians review EHR vignettes and related questions, proposed answers, and distractor “trap” answers across diagnosis and treatment tasks covering cardiovascular, nervous, hematologic, respiratory, digestive, urinary, reproductive, musculoskeletal, and integumentary systems. They assess benchmark-item quality, including vignette accuracy and completeness, whether a vignette gives away the answer, answer correctness and gradeability, and trap quality. They also assess whether reasoning follows from the vignette facts, check reasoning traces, and write short rationales. Work includes three independent blind reads followed by physician adjudication.

Contributors choose when and how much to work. Tasks are estimated at around 10–20 hours per week during active project phases; this is neither a minimum nor guaranteed workload. Payment is per approved task at a flat rate set to about $100 per estimated hour, with QA and adjudication tasks paid at a higher rate. A $50 one-time onboarding reward is paid after the first production task is approved. Additional incentives may be available for high-quality approved tasks, under terms shared in the project guidelines. Participation follows the process: apply, pass qualification(s), join a project, complete tasks, and get paid.

Requirements

• Medical degree (MD or DO).
• Active, unrestricted US medical license, verified at onboarding and re-checked monthly.
• Board certification or completed residency training; recent direct patient care, with attending-level experience preferred.
• Broad, multi-system diagnostic and treatment experience in internal, family, emergency, or hospital medicine.
• Strong clinical reasoning, including differential diagnosis, next-step management, and application of evidence-based guidelines.
• Strong written English (C1+).

Nice To Haves

• Prior experience in medical AI evaluation, clinical content or exam-question review, or medical annotation/QA.

Responsibilities

• Evaluate clinical EHR vignettes paired with questions, proposed answers, and distractor “trap” answers across diagnosis and treatment tasks.
• Assess benchmark items for vignette accuracy and completeness, whether the vignette gives away the answer, answer correctness and gradeability, and trap quality.
• Check whether reasoning reaches the answer using vignette facts alone, review reasoning traces, and write short rationales.
• Participate in three independent blind reads followed by physician adjudication.
• Submit tasks within the project timeline and meet the listed acceptance criteria.

Benefits

• $50 one-time onboarding reward, paid once the first production task is approved.
• Additional earnings may be available for high-quality approved tasks.
• Contributors can choose when and how much to work, with tasks designed to fit around clinical practice.
• Contribute anytime, from anywhere.
• No patient contact and no malpractice exposure.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: Ph.D. or professional degree