Physician (MD/DO) - Medical AI Evaluation
About The Position
Mindrift connects specialists with project-based AI opportunities for leading technology companies, focused on testing, evaluating, and improving AI systems. This project is for actively practicing physicians with an active, unrestricted US medical license who live outside the US. General and internal medicine are the main focus. Physicians evaluate clinical EHR vignettes for a medical AI evaluation project, with no patient contact or malpractice exposure. Participation is project-based and designed to fit around a clinical practice. Contributors choose when and how much to work; tasks are estimated at around 10–20 hours per week during active project phases, but this is an estimate, not a minimum or guaranteed workload. Tasks must be completed within the project timeline and meet the acceptance criteria to be approved. Contributors review clinical vignettes paired with questions, proposed answers, and distractor “trap” answers across diagnosis and treatment tasks. They assess benchmark-item quality, including vignette accuracy and completeness, whether a vignette gives away its answer, answer correctness and gradeability, and trap quality. They check whether the reasoning chain reaches the answer using only the vignette facts, correct reasoning traces, and write short rationales. Work includes three independent blind reads followed by physician adjudication. Payment is a flat rate per approved task, set at an estimated equivalent of about $100 per hour based on task time, with no tiers or speed adjustments to the base rate. QA and adjudication tasks are paid at a higher rate. A one-time $50 onboarding reward is paid after the first production task is approved, and additional earnings may be available for high-quality approved tasks under terms described in the project guidelines. Applicants should submit a CV in English and indicate their English proficiency. The process is: apply, pass qualification(s), join a project, complete tasks, and get paid.
Requirements
- Medical degree (MD or DO).
- Active, unrestricted US medical license; license is verified at onboarding and re-checked monthly.
- Actively practicing physician living outside the US.
- Board certification or completed residency training, with recent direct patient care.
- Broad, multi-system diagnostic and treatment experience in internal, family, emergency, or hospital medicine.
- Strong clinical reasoning, including differential diagnosis, next-step management, and application of evidence-based guidelines.
- Strong written English (C1+).
Nice To Haves
- Attending-level clinical experience is preferred.
- Prior experience in medical AI evaluation, clinical content or exam-question review, or medical annotation/QA.
Responsibilities
- Evaluate clinical EHR vignettes paired with questions, proposed answers, and distractor trap answers across diagnosis and treatment tasks.
- Score benchmark items for vignette accuracy and completeness, whether the vignette gives away the answer, answer correctness and gradeability, and trap quality.
- Check whether the reasoning chain reaches the answer from vignette facts alone, assess reasoning traces, and write short rationales.
- Participate in three independent blind reads followed by physician adjudication.
- Complete and submit tasks within the project timeline and meet the applicable acceptance criteria.
Benefits
- $50 one-time onboarding reward after the first production task is approved.
- Additional earnings for high-quality approved tasks.
- QA and adjudication tasks paid at a higher rate.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
Education Level
Ph.D. or professional degree
