Physician (MD/DO) - Medical AI Evaluation

Mindrift • Lebanon, PA

2d • Remote

About The Position

Mindrift is seeking US-licensed physicians (MD/DO) to evaluate clinical EHR vignettes for a medical AI evaluation project. General and internal medicine are the main focus. This is a task-based, independent contractor opportunity limited to this project, not an employment relationship. The work involves reviewing written clinical cases and does not involve patient contact or malpractice exposure.

Contributors assess clinical cases against project acceptance criteria. Tasks may involve evaluating proposed answers and distractor answers across multiple body systems, reviewing the quality of clinical reasoning and benchmark items, checking whether conclusions follow from the vignette facts, and writing short rationales. Each case receives three independent blind reads followed by physician adjudication.

The work is fully self-scheduled, with no minimum hours, set schedule, or required availability. Task availability varies with project phases, and contributors choose which tasks to complete. Earnings are paid per completed, approved task at a fixed rate for each task type; QA and adjudication tasks have higher per-task rates. A one-time $50 onboarding reward is paid after the first task is approved, and quality incentives may provide additional earnings under the project guidelines.

Requirements

• Medical degree (MD or DO).
• Active, unrestricted US medical license, verified at onboarding and re-checked monthly.
• Strong written English proficiency (C1+).
• Strong clinical reasoning skills, including differential diagnosis, next-step management, and application of evidence-based guidelines.
• Board certification or completed residency training, with direct patient care experience.

Nice To Haves

• Attending-level experience is preferred.
• Broad, multi-system diagnostic and treatment experience in internal medicine, family medicine, emergency medicine, or hospital medicine.
• Prior experience in medical AI evaluation, clinical content or exam-question review, or medical annotation/QA.
• Practicing or retired physicians are welcome.

Responsibilities

• Evaluate clinical EHR vignettes paired with a question, proposed answer, and distractor or trap answer across diagnosis and treatment tasks.
• Review cases spanning cardiovascular, nervous, hematologic, respiratory, digestive, urinary, reproductive, musculoskeletal, and integumentary systems.
• Score benchmark items for vignette accuracy and completeness, whether the vignette gives away the answer, answer correctness and gradeability, and trap quality.
• Assess whether the reasoning chain reaches the answer using only facts in the vignette and whether the reasoning trace is correct.
• Write short rationales for evaluations.
• Complete and submit tasks through the platform in accordance with project acceptance criteria.

Benefits

• Fully self-scheduled work with no minimum hours, set schedule, or required availability.
• Fixed rate per approved task, set in advance.
• One-time $50 onboarding reward after the first task is approved.
• Additional quality incentives for high-quality approved tasks under project guidelines.
• Higher per-task rates for QA and adjudication tasks.

What This Job Offers

• Education Level: Ph.D. or professional degree