Physician (MD/DO) - Medical AI Evaluation

Mindrift

2d • Remote

About The Position

Mindrift is seeking US-licensed physicians to evaluate clinical EHR vignettes for a medical AI evaluation project. The work involves reviewing written clinical cases and assessing them against project acceptance criteria; there is no patient contact or malpractice exposure. The project focuses primarily on general and internal medicine and includes cases across cardiovascular, nervous, hematologic, respiratory, digestive, urinary, reproductive, musculoskeletal, and integumentary systems.

Contributors assess vignette accuracy and completeness, whether a vignette gives away its answer, the correctness and gradeability of proposed answers, and the quality of distractor “trap” answers. They also assess whether reasoning chains reach answers using only the vignette facts, review reasoning traces, and write short rationales. Each case receives three independent blind reads followed by physician adjudication.

This is a task-based, time-limited independent contractor opportunity for the project only, not an employment relationship. Contributors choose which tasks to take and when to complete them. There are no minimum hours, set schedules, or required availability, and task availability varies by project phase. Tasks are submitted through the platform and paid only when completed and approved, at fixed rates set by task type. QA and adjudication tasks have higher per-task rates. A one-time $50 onboarding reward is paid after the contributor’s first task is approved; additional quality incentives may be offered under the project guidelines.

Requirements

• Medical degree (MD or DO).
• Active, unrestricted US medical license; license is verified at onboarding and re-checked monthly.
• Strong clinical reasoning, including differential diagnosis, next-step management, and application of evidence-based guidelines.
• Strong written English (C1+).
• Must be a physician; the project is not suitable for medical students, unlicensed medical graduates, non-physician clinicians, or non-clinical healthcare titles.

Nice To Haves

• Board certification or completed residency training and direct patient care experience; attending-level experience is preferred.
• Broad, multi-system diagnostic and treatment experience in internal, family, emergency, or hospital medicine.
• Prior experience in medical AI evaluation, clinical content or exam-question review, or medical annotation/QA.
• Practising or retired physicians are welcome.

Responsibilities

• Evaluate clinical EHR vignettes paired with questions, proposed answers, and distractor “trap” answers across diagnosis and treatment tasks.
• Assess vignette accuracy and completeness, including whether the vignette gives away the answer.
• Evaluate answer correctness and gradeability and assess the quality of distractor answers.
• Determine whether the reasoning chain reaches the answer using only facts in the vignette and review the correctness of reasoning traces.
• Write short rationales for evaluations.
• Complete and submit project tasks through the platform in accordance with the project’s acceptance criteria.

Benefits

• $50 one-time onboarding reward after the first task on the project is approved.
• Additional quality incentives for high-quality approved tasks, on terms set out in the project guidelines.
• Fully self-scheduled work with no minimum hours, set schedule, or required availability.
• Fixed rate for each task, set in advance; QA and adjudication tasks have a higher per-task rate.

What This Job Offers

• Career Level: Senior
• Education Level: No Education Listed