Physician (MD/DO) - Medical AI Evaluation
About The Position
Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. Participation is project-based. We’re looking for US-based, actively practicing physicians (MD/DO) to evaluate clinical EHR vignettes for a medical AI evaluation program. General and internal medicine are our main focus. While each project involves unique tasks, contributors may evaluate clinical EHR vignettes paired with a question, a proposed answer, and a distractor “trap” answer across diagnosis and treatment tasks, spanning cardiovascular, nervous, hematologic, respiratory, digestive, urinary, reproductive, musculoskeletal, and integumentary systems; score the clinical reasoning quality of benchmark items: vignette accuracy and completeness, whether the vignette gives the answer away, answer correctness and gradeability, and trap quality; check whether the reasoning chain reaches the answer from vignette facts alone, correct reasoning traces, and write short rationales; and work within three independent blind reads, followed by physician adjudication. Real clinical complexity only. You’re improving the AI tools you’ll eventually use yourself. If you’re a practicing physician ready to take on this challenging and engaging project, join us!
Requirements
- Medical degree (MD or DO)
- Active, unrestricted US medical license (verified with the issuing medical board)
- Board certification or completed residency training, with recent direct patient care (attending-level preferred)
- Broad, multi-system diagnostic and treatment experience (internal, family, emergency, or hospital medicine)
- Strong clinical reasoning: differential diagnosis, next-step management, and application of evidence-based guidelines
- Strong written English (C1+)
Nice To Haves
- Prior experience in medical AI evaluation, clinical content or exam-question review, or medical annotation/QA
Responsibilities
- Evaluate clinical EHR vignettes paired with a question, a proposed answer, and a distractor “trap” answer across diagnosis and treatment tasks, spanning cardiovascular, nervous, hematologic, respiratory, digestive, urinary, reproductive, musculoskeletal, and integumentary systems
- Score the clinical reasoning quality of benchmark items: vignette accuracy and completeness, whether the vignette gives the answer away, answer correctness and gradeability, and trap quality
- Check whether the reasoning chain reaches the answer from vignette facts alone, correct reasoning traces, and write short rationales
- Work within three independent blind reads, followed by physician adjudication
Benefits
- Paid per accepted task
- Rate depends on qualification tier and efficiency
- Up to the equivalent of $35/hr
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Part-time
Career Level
Mid Level
