Licensed Physician (MD/DO) - Freelance AI Trainer

Mindrift • Duluth, MN

5d • Remote

About The Position

Mindrift connects specialists with project-based AI opportunities for leading technology companies, focused on testing, evaluating, and improving AI systems. This project is for actively practicing physicians with an active, unrestricted US medical license to evaluate clinical EHR vignettes for a medical AI evaluation project, primarily focused on general and internal medicine. The work involves reviewing written clinical cases and has no patient contact or malpractice exposure. All reviewers are license-verified physicians.

Contributors evaluate clinical vignettes paired with questions, proposed answers, and distractor “trap” answers across diagnosis and treatment tasks in multiple body systems. They assess benchmark-item quality, including vignette accuracy and completeness, whether a vignette gives away its answer, answer correctness and gradeability, and trap quality. They also check reasoning chains, correct reasoning traces, and write short rationales. The process includes three independent blind reads followed by physician adjudication.

The opportunity is project-based and designed to fit around clinical practice. Contributors choose when and how much to work, from anywhere. Tasks are estimated at about 10–20 hours per week during active project phases; this is not a minimum or guaranteed workload. Tasks must be submitted within the project timeline and meet acceptance criteria to be approved. Payment is per approved task at a flat rate estimated to be about $100 per hour based on task time, with higher rates for QA and adjudication tasks. A one-time $50 onboarding reward is paid after the first production task is approved, and additional incentives may be available for high-quality approved tasks under project guidelines.

Requirements

• Medical degree (MD or DO).
• Active, unrestricted US medical license, verified at onboarding and re-checked monthly.
• Board certification or completed residency training.
• Recent direct patient care experience; attending-level experience is preferred.
• Broad, multi-system diagnostic and treatment experience in internal, family, emergency, or hospital medicine.
• Strong clinical reasoning, including differential diagnosis, next-step management, and application of evidence-based guidelines.
• Strong written English proficiency (C1+).

Nice To Haves

• Prior experience in medical AI evaluation, clinical content or exam-question review, or medical annotation/QA.
• Attending-level clinical experience.

Responsibilities

• Evaluate clinical EHR vignettes paired with questions, proposed answers, and distractor trap answers across diagnosis and treatment tasks and multiple body systems.
• Score the clinical reasoning quality of benchmark items, including vignette accuracy and completeness, whether the vignette reveals the answer, answer correctness and gradeability, and trap quality.
• Check whether reasoning chains reach answers using only facts from the vignette, review reasoning traces, and write short rationales.
• Participate in a workflow of three independent blind reads followed by physician adjudication.
• Submit tasks within the project timeline and meet the stated acceptance criteria.

Benefits

• Flexible, project-based work that can be fitted around clinical practice.
• Work from anywhere and choose when and how much to contribute.
• No patient contact or malpractice exposure.
• Flat-rate payment per approved task, estimated at about $100 per hour based on task time.
• Higher rate for QA and adjudication tasks.
• One-time $50 onboarding reward after the first production task is approved.
• Additional incentives for high-quality approved tasks.

What This Job Offers

• Career Level: Senior
• Education Level: Ph.D. or professional degree