Director, Clinical AI Research & Evaluation

AstraZeneca•Cambridge, MA

2d•$186,014 - $279,021•Hybrid

About The Position

AstraZeneca’s AI Science & Innovation (AISI), within the Enterprise AI Unit, builds and delivers AI models, agents, platforms, and applications across R&D. The Director, Clinical AI Research & Evaluation will lead the scientific foundation for trustworthy, measurable clinical AI systems that can scale into regulated practice. The role is part of the Clinical AI Applied Science & Translation team and is based in Boston, Massachusetts. The initial focus is an agentic system intended to reduce administrative work for research associates at trial sites, with methods and assets expected to extend across the Clinical Development AI portfolio. This is a research leadership role with delivery accountability. The Director will set the research agenda for evaluating clinical AI, studying human interaction with AI in clinical workflows, and establishing the evidence standards needed for deployment. The role directs scientists and engineers, partners with product owners and Engineering, and develops evaluation and human-factors practices that support responsible deployment. The Director will define evaluation methodology and assets, investigate clinical failure modes, and establish readiness criteria for movement from prototype to pilot and deployment, including transition to regulated use. The role also collaborates with Quality, Regulatory, Privacy, Security, and R&D IT to ensure evidence meets GxP, data integrity, and inspection-readiness requirements. The position is expected to contribute to external scientific credibility and relevant industry standards. The role follows an office-working expectation of an average minimum of three days per week, balanced with individual flexibility. The annual base pay range is $186,014 to $279,021 USD. Positions may be eligible for short-term incentive bonuses and equity-based awards for salaried roles. Benefits may include qualified retirement programs, paid time off, and health, dental, and vision coverage, subject to applicable plan terms.

Requirements

PhD in Machine Learning, Computer Science, Human-Computer Interaction, Statistics, Biomedical Informatics, or a related quantitative discipline; alternatively, an MD or equivalent experience with a research record demonstrating comparable rigor.
At least 8 years of post-PhD or equivalent experience in AI/ML research, evaluation science, or human factors in healthcare, life sciences, clinical research, or another safety-critical domain.
Deep expertise in AI/ML evaluation methodology, including experimental design, statistical inference, uncertainty quantification, measurement reliability, and evaluation of LLM-based or multi-step agentic workflows.
Demonstrated research leadership, including setting a scientific agenda, directing a team, producing research outputs, and translating methodology into practical, shipping systems.
Current hands-on coding ability in Python and SQL, including evaluation pipelines, data analysis, statistical testing, and version-controlled collaborative development.
Strong grounding in human-computer interaction or human factors and application to AI-assisted decision-making in high-stakes environments.
Excellent communication across technical, clinical, quality, regulatory, and executive audiences, including the ability to represent uncertainty and residual risk accurately.
Demonstrated ability to lead cross-functional work through influence and make go/no-go recommendations under ambiguity.
Experience with LLM evaluation platforms and LLMOps/MLOps and observability for deployed AI systems.

Nice To Haves

Direct experience with clinical trial conduct, site monitoring, risk-based monitoring, medical monitoring, or clinical data management in a pharmaceutical, biotech, or CRO environment.
Experience producing audit-ready evidence under GxP requirements, including model risk management, audit trails, traceability, and inspection readiness for regulated AI systems.
Experience designing or conducting prospective validations, implementation studies, or human-computer interaction studies in live clinical or healthcare workflows.
Peer-reviewed publications, standards-body contributions, open-source benchmarks, or regulatory and industry consortium engagement in AI evaluation, human factors for AI, or clinical AI governance.
Familiarity with causal inference, calibration, subgroup analysis, weak supervision, active learning, or evaluation methods for learning with imperfect labels.
Experience working with Quality or regulatory functions on computerized system validation, validation plans, or submission-relevant AI evidence.

Responsibilities

Define and own scientific methodology for evaluating clinical AI systems, including evaluation harnesses, benchmark construction, failure-mode taxonomies, red-team protocols, and lifecycle monitoring strategies.
Build a structured catalog of clinical system failure modes and conduct structured testing before deployment.
Advance evaluation methods for noisy or delayed ground truth, expert disagreement, multi-step agentic workflows, human-in-the-loop performance, automation bias, and other settings where simple metrics are inadequate.
Develop human-factors and workflow-integration methods to assess how AI outputs shape human judgment, effective oversight, trust calibration, and patient-safety review.
Publish and communicate findings externally where appropriate.
Design and implement evaluation frameworks for progression from prototype to controlled pilot, pilot to deployment, and non-GxP to regulated use.
Help define human-review requirements for user-facing AI outputs, including sign-off requirements and oversight structures in validation plans.
Build reusable evaluation assets, including test sets, simulation environments, scoring rubrics, regression suites, and analysis pipelines.
Lead hands-on analysis of failures such as fabrications or falsifications, unsupported claims, data leakage, subgroup performance issues, and unsafe workflow behavior.
Lead and contribute to a team of scientists and engineers focused on evaluation, human factors, and responsible deployment.
Work with Quality, Regulatory, Privacy, Security, and R&D IT to ensure evidence meets GxP, data-integrity, and inspection-readiness requirements.
Advise product owners and leadership on evidence-based go/no-go decisions and communicate residual risk clearly.
Create reusable evaluation standards and clinical AI governance practices for the Clinical Development portfolio and contribute to broader industry standards.
Represent the team’s scientific position in external forums, standards bodies, and relevant regulatory engagements.