Remote | Medical & Health Sciences Benchmark Expert — $85–$110/hour

24-Mag•New York, NY

1d•Remote

About The Position

We are sharing a specialised part-time consulting opportunity for physicians, medical researchers, and health science professionals with advanced expertise in clinical medicine, diagnostics, biomedical research, or healthcare delivery. This role supports an AI research initiative focused on developing rigorous benchmarks across medicine and health sciences. Selected experts will author or verify challenging multiple-choice assessment content, evaluate clinical and scientific accuracy, develop well-supported reference solutions, and help establish high-quality evaluation standards for advanced AI systems.

Requirements

MD, DO, PhD, or current doctoral candidacy in Medicine, Biomedical Sciences, Public Health, or a closely related health discipline
Master's degree may be considered for candidates with exceptional expertise in a relevant subdomain
Strong command of graduate-level medical knowledge, clinical reasoning, and biomedical research methodology
Expertise in one or more areas including Clinical Medicine, Surgery, Medical Imaging, Diagnostics, Pharmacovigilance, Healthcare Management, Health Economics, Rehabilitation, or Allied Health
Strong ability to evaluate clinical evidence and interpret biomedical research
Excellent written English and ability to communicate complex medical concepts clearly and precisely

Nice To Haves

Board certification or substantial clinical experience is highly valued
Academic or clinical research publications are highly valued
University-level teaching, examination development, or medical assessment experience is advantageous
Strong attention to clinical accuracy, evidence quality, and methodological rigour

Responsibilities

Create original multiple-choice questions that test deep medical and clinical understanding rather than surface-level recall
Develop challenging problems within relevant areas of medical or health science expertise
Ensure questions are self-contained, precise, unambiguous, and independently solvable
Design questions appropriate for undergraduate, advanced undergraduate, and postgraduate difficulty levels
Produce one correct answer alongside nine plausible and carefully constructed distractors
Review pre-written medical and health science questions for accuracy, clarity, rigour, and completeness
Identify clinical inaccuracies, ambiguity, unsupported assumptions, or solvability issues
Edit questions and answer choices where required
Verify that designated correct answers are supported by current medical knowledge and appropriate evidence
Document substantive changes and explain the clinical or scientific reasoning behind them
Develop assessment content involving diagnosis, treatment, clinical decision-making, and patient management
Create problems requiring integration of symptoms, examination findings, laboratory results, and clinical history
Evaluate diagnostic reasoning and differential diagnosis
Apply evidence-based principles to management and treatment scenarios
Develop clinically realistic questions across medical and surgical disciplines
Create questions involving diagnostic imaging, laboratory testing, and clinical investigation
Evaluate interpretation of imaging findings and diagnostic data
Develop problems requiring appropriate selection and sequencing of diagnostic tests
Assess sensitivity, specificity, predictive value, and other diagnostic-performance concepts where relevant
Apply clinical reasoning to complex diagnostic scenarios
Develop content involving adverse drug reactions, medication safety, and pharmacovigilance
Evaluate causality, signal detection, risk management, and safety-monitoring scenarios
Create questions involving benefit-risk assessment and post-market surveillance
Apply appropriate clinical and regulatory reasoning to medication-safety problems
Evaluate interpretation of safety data and pharmacovigilance evidence
Develop assessment content involving healthcare systems, resource allocation, and health economics
Evaluate questions involving quality, access, efficiency, and healthcare delivery
Apply economic and organisational principles to healthcare-management scenarios
Develop problems involving cost-effectiveness, outcomes, and system-level decision-making
Integrate clinical and operational perspectives where appropriate
Create questions involving rehabilitation medicine and allied health disciplines
Develop scenarios addressing functional assessment, recovery, and multidisciplinary care
Evaluate treatment planning and patient-centred rehabilitation approaches
Apply evidence-based reasoning to physiotherapy, occupational therapy, and related domains
Assess clinical outcomes and functional measures where relevant
Provide 1–5 academic or clinical references per question where required
Use reputable sources such as peer-reviewed journals, clinical guidelines, professional societies, and academic institutions
Develop clear, structured solution explanations supporting the correct answer
Rate question difficulty according to defined Medium, Hard, and Expert standards
Help maintain consistent clinical and scientific quality across benchmark content
Contribute to gold-standard evaluation materials used to assess advanced AI capabilities