Physician Reviewer (MD/DO) - Medical AI Evaluation

Mindrift • Reading, PA

2d • Remote

About The Position

Mindrift connects specialists with project-based AI opportunities for leading technology companies. This is a task-based, independent contractor opportunity—not an employment relationship—and participation is limited to this project.

Physician Reviewers with an active, unrestricted U.S. medical license evaluate written clinical EHR vignettes for a medical AI evaluation project, primarily focused on general and internal medicine. The work involves reviewing cases against project acceptance criteria, scoring benchmark-item quality, checking clinical reasoning, and writing short rationales. There is no patient contact or malpractice exposure. Each case receives three independent blind reads followed by physician adjudication.

Contributors choose which tasks to take and when to complete them. There are no minimum hours, set schedules, required availability, time tracking, or hour logging; task availability varies by project phase. Payment is a fixed rate per completed and approved task, with higher per-task rates for QA and adjudication tasks. Quality incentives may provide additional earnings under the project guidelines. A one-time $50 onboarding reward is paid after the first task is approved.

Requirements

• Medical degree (MD or DO).
• Active, unrestricted U.S. medical license, verified at onboarding and re-checked monthly.
• Board certification or completed residency training, with direct patient care experience; attending-level experience preferred.
• Broad, multi-system diagnostic and treatment experience, such as internal, family, emergency, or hospital medicine.
• Strong clinical reasoning, including differential diagnosis, next-step management, and application of evidence-based guidelines.
• Strong written English (C1+).

Nice To Haves

• Prior experience in medical AI evaluation, clinical content or exam-question review, or medical annotation/quality assurance.
• Attending-level experience.

Responsibilities

• Evaluate written clinical EHR vignettes paired with a question, proposed answer, and distractor or “trap” answer across diagnosis and treatment tasks and multiple body systems.
• Score benchmark items for vignette accuracy and completeness, whether the vignette gives away the answer, answer correctness and gradeability, and trap quality.
• Assess whether the reasoning chain reaches the answer using only facts in the vignette, check reasoning traces, and write short rationales.
• Complete tasks through the platform in accordance with project acceptance criteria.

Benefits

• $50 one-time onboarding reward after the first task is approved.
• Additional earnings for high-quality approved tasks, under terms set out in the project guidelines.
• Flexible, self-scheduled task participation with no minimum hours or required availability.

What This Job Offers

• Career Level: Senior
• Education Level: Ph.D. or professional degree