Licensed Physician (MD/DO) - Freelance AI Trainer

Mindrift • New York, NY

6d • Remote

About The Position

Mindrift connects specialists with project-based AI opportunities for leading technology companies, focused on testing, evaluating, and improving AI systems. This is a task-based, independent-contractor opportunity limited to this project, not an employment relationship. It is remote for physicians licensed in eligible US states, with no patient contact or malpractice exposure. Contributors review written clinical cases, and all contributors are license-verified physicians.

Physicians evaluate clinical EHR vignettes for a medical AI evaluation project, primarily focused on general and internal medicine. Each task is a written clinical case assessed against project acceptance criteria. Contributors assess benchmark items for clinical reasoning quality, including vignette accuracy and completeness, whether the vignette reveals the answer, answer correctness and gradeability, and the quality of trap answers. They also check whether reasoning follows from vignette facts, assess reasoning traces, and write short rationales. Cases receive three independent blind reads followed by physician adjudication.

The work is fully self-scheduled, with no minimum hours, set schedule, required availability, time tracking, or hour logging. Task availability varies by project phase, and contributors choose which tasks to take and when. Earnings are paid only for completed, approved tasks, with a fixed rate set in advance for each task type. QA and adjudication tasks have a higher per-task rate. A one-time $50 onboarding reward is paid after the first task is approved, and quality incentives may provide additional earnings under the project guidelines.

Requirements

• Medical degree (MD or DO).
• Active, unrestricted US medical license; license is verified at onboarding and re-checked monthly.
• Board certification or completed residency training, with direct patient care experience; attending-level experience is preferred.
• Broad, multi-system diagnostic and treatment experience in internal, family, emergency, or hospital medicine.
• Strong clinical reasoning, including differential diagnosis, next-step management, and application of evidence-based guidelines.
• Strong written English at C1 or higher.

Nice To Haves

• Prior experience in medical AI evaluation, clinical content or exam-question review, or medical annotation/quality assurance.
• Attending-level experience.

Responsibilities

• Evaluate written clinical EHR vignettes paired with questions, proposed answers, and distractor trap answers across diagnosis and treatment tasks and multiple medical systems.
• Score clinical reasoning quality, including vignette accuracy and completeness, whether a vignette gives away the answer, answer correctness and gradeability, and trap quality.
• Check that reasoning chains reach answers using only the facts in the vignette and assess the correctness of reasoning traces.
• Write short rationales for evaluations.
• Complete and submit project tasks through the platform in accordance with the project’s acceptance criteria.

Benefits

• $50 one-time onboarding reward after the first task is approved.
• Additional quality incentives for high-quality approved tasks, on terms set out in the project guidelines.

What This Job Offers

• Career Level: Senior
• Education Level: Ph.D. or professional degree