Physician (MD/DO) - Medical AI Evaluation

Mindrift

2d • Remote

About The Position

Mindrift connects specialists with project-based AI opportunities for leading technology companies, focused on testing, evaluating, and improving AI systems. This is a task-based, independent contractor opportunity limited to this project, not an employment relationship. It is remote and open to US-licensed physicians based in eligible US states. There is no patient contact or malpractice exposure, and contributors review written clinical cases. Participation is fully self-scheduled, with no minimum hours or set schedule; contributors choose which tasks to take and when. Task availability varies by project phase.

Physicians with an active, unrestricted US medical license will evaluate clinical EHR vignettes for a medical AI evaluation project, primarily in general and internal medicine. Each task is a written clinical case assessed against project acceptance criteria. Work may include reviewing vignettes paired with questions, proposed answers, and distractor “trap” answers across multiple medical systems; evaluating clinical reasoning and benchmark-item quality; checking whether answers and reasoning traces follow from vignette facts; and writing short rationales. Cases receive three independent blind reads followed by physician adjudication.

Tasks are submitted through the platform and paid only when completed and approved. Each task type has a fixed rate set in advance, with higher per-task rates for QA and adjudication tasks. There is no time tracking, hour logging, tiering, or speed-based rate adjustment. A one-time $50 onboarding reward is paid after the contributor’s first task is approved, and additional quality incentives may be available under the project guidelines.

Requirements

• Medical degree (MD or DO).
• Active, unrestricted US medical license, verified at onboarding and re-checked monthly.
• Strong written English (C1+).
• Must be a physician; the project is not suitable for medical students, unlicensed medical graduates, non-physician clinicians, or non-clinical healthcare titles.

Nice To Haves

• Board certification or completed residency training, with direct patient care experience; attending-level experience preferred.
• Broad, multi-system diagnostic and treatment experience in internal, family, emergency, or hospital medicine.
• Strong clinical reasoning, including differential diagnosis, next-step management, and application of evidence-based guidelines.
• Prior experience in medical AI evaluation, clinical content or exam-question review, or medical annotation/QA.
• Practising or retired physicians are welcome.

Responsibilities

• Evaluate clinical EHR vignettes paired with questions, proposed answers, and distractor “trap” answers across diagnosis and treatment tasks and a range of medical systems.
• Score benchmark items for vignette accuracy and completeness, whether the vignette gives away the answer, answer correctness and gradeability, and trap quality.
• Check whether the reasoning chain reaches the answer using only facts from the vignette.
• Correct reasoning traces and write short rationales.
• Submit tasks through the platform for review against project acceptance criteria.

Benefits

• Remote work.
• Fully self-scheduled participation with no minimum hours, set schedule, or required availability.
• Fixed per-task rates, with higher per-task rates for QA and adjudication tasks.
• $50 one-time onboarding reward after the first task is approved.
• Additional earnings for high-quality approved tasks under project guidelines.

What This Job Offers

• Education Level: Ph.D. or professional degree