Physician Reviewer (MD/DO) - Medical AI Evaluation

Mindrift • Sitka, AK

1d • Remote

About The Position

Mindrift is seeking actively practicing physicians (MD or DO) with an active, unrestricted US medical license to evaluate clinical EHR vignettes for a medical AI evaluation project, with a primary focus on general and internal medicine. The work involves reviewing written clinical cases only; there is no patient contact or malpractice exposure. Contributors evaluate vignettes and proposed answers across diagnosis and treatment tasks covering cardiovascular, nervous, hematologic, respiratory, digestive, urinary, reproductive, musculoskeletal, and integumentary systems. They assess clinical reasoning, vignette quality, answer correctness and gradeability, and the quality of distractor “trap” answers; verify reasoning traces; and provide short rationales. Work includes three independent blind reads followed by physician adjudication.

This is a project-based opportunity designed to fit around clinical practice. Contributors choose when and how much to work, from anywhere. Tasks are estimated at approximately 10–20 hours per week during active project phases, but this is neither a minimum nor a guaranteed workload. Tasks must be submitted within project timelines and meet acceptance criteria. Pay is per approved task at a flat rate set to approximately $100 per hour equivalent based on estimated task time. QA and adjudication tasks are paid at a higher rate, and high-quality approved tasks may receive additional incentives. A one-time $50 onboarding reward is paid after the first production task is approved.

Requirements

• Medical degree (MD or DO).
• Active, unrestricted US medical license, verified at onboarding and re-checked monthly.
• Board certification or completed residency training.
• Recent direct patient care; attending-level experience is preferred.
• Broad, multi-system diagnostic and treatment experience in internal, family, emergency, or hospital medicine.
• Strong clinical reasoning skills, including differential diagnosis, next-step management, and application of evidence-based guidelines.
• Strong written English proficiency (C1+).

Nice To Haves

• Prior experience in medical AI evaluation, clinical content or exam-question review, or medical annotation/quality assurance.
• Attending-level clinical experience.

Responsibilities

• Evaluate clinical EHR vignettes paired with questions, proposed answers, and distractor “trap” answers across diagnosis and treatment tasks.
• Assess vignette accuracy and completeness, whether a vignette reveals the answer, answer correctness and gradeability, and distractor quality.
• Determine whether reasoning reaches the answer using only the facts in the vignette, check reasoning traces, and write short rationales.
• Participate in three independent blind reads followed by physician adjudication.
• Submit tasks within project timelines and meet the applicable acceptance criteria.

Benefits

• Flexible, self-directed schedule and contribution level.
• Work from anywhere.
• No patient contact and no malpractice exposure.
• Approximately $100/hour equivalent flat rate per approved task, based on estimated task time.
• Higher pay rates for QA and adjudication tasks.
• Additional earnings may be available for high-quality approved tasks.
• One-time $50 onboarding reward after the first production task is approved.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: Ph.D. or professional degree