Clinical Content Specialist – Clinical AI Evaluation Specialist

Wolters Kluwer•Waltham, MA
•$85,600 - $149,400•Hybrid

About The Position

Wolters Kluwer is seeking a Clinical AI Evaluation Specialist to help ensure AI-generated clinical content is accurate, evidence-based, relevant, and useful in practice. The role conducts systematic evaluations of AI outputs grounded in trusted clinical content, applying evidence-based medicine principles, analytical methods, and professional judgment to assess performance, identify patterns, and generate insights that inform system improvement. This is a professional individual-contributor role within the Clinical AI team. The Evaluation Specialist independently executes defined evaluation work within the strategy and standards established by Clinical AI leadership. The role is suited to an individual with graduate training and experience in evidence-based medicine, clinical content development, research methodology, public health, or a related discipline. The position is hybrid, with eight days a month in the Waltham, Massachusetts office.

Requirements

  • PhD, MPH, or a related graduate degree in clinical epidemiology, public health, health services research, biomedical sciences, clinical informatics, evidence-based medicine, or a related field.
  • Experience in evidence-based medicine, clinical content development, clinical research, systematic evidence assessment, or a related area.
  • Strong analytical skills, including the ability to apply structured evaluation methods, interpret complex findings, and identify meaningful patterns.
  • Excellent written and verbal communication skills, including the ability to explain findings clearly to clinical, editorial, product, and technical audiences.
  • Experience translating evidence or research findings into structured clinical content.
  • Strong attention to detail and commitment to methodological rigor.
  • Ability to independently execute defined work while collaborating effectively across disciplines.
  • Comfort working in an evolving environment with incomplete information and iterative development cycles.
  • Ability to distinguish substantive clinical or evidence problems from issues of wording, organization, or presentation.

Nice To Haves

  • Training or applied experience in statistics, biostatistics, epidemiology, quantitative research methods, or data analysis.
  • Experience evaluating generative AI, machine-learning systems, clinical decision-support tools, or other digital health products.
  • Familiarity with AI evaluation methods, annotation workflows, benchmark development, error taxonomies, or human-in-the-loop validation.
  • Experience working with medical editors, clinical experts, product teams, engineers, or data scientists.
  • Familiarity with prompt design, retrieval-augmented generation, clinical search, or content-grounded AI systems.
  • Knowledge of patient safety, healthcare quality, or clinical risk principles.

Responsibilities

  • Conduct structured evaluations of AI-generated clinical content.
  • Develop test cases, evaluation criteria, scoring rubrics, and annotation guidance for defined use cases.
  • Evaluate outputs for evidence fidelity, clinical accuracy, relevance, completeness, clarity, and appropriate expression of uncertainty.
  • Apply consistent analytical methods and professional judgment to evaluation decisions.
  • Identify recurring errors, performance gaps, and patterns across clinical topics and question types.
  • Conduct quantitative and qualitative analyses of evaluation results.
  • Document evaluation methods, findings, limitations, and conclusions clearly and reproducibly.
  • Evaluate system performance over time and assess the impact of system changes.
  • Apply evidence-based medicine principles to determine how accurately AI outputs represent source content and supporting evidence.
  • Assess whether outputs appropriately synthesize, prioritize, and communicate clinical information.
  • Contribute to evaluation frameworks that reflect the needs of clinicians using AI-supported clinical decision tools.
  • Review clinical content and source materials to develop test cases and inform evaluation decisions.
  • Collaborate with clinical experts when evaluation requires specialized clinical judgment.
  • Translate evaluation findings into clear, actionable insights for Clinical AI, product, engineering, and editorial teams.
  • Contribute evidence and analysis to the refinement of prompts, retrieval approaches, clinical search logic, and content-grounding methods.
  • Inform product and system-development decisions through rigorous evaluation findings.
  • Participate in focused red-teaming, safety testing, and validation activities.
  • Analyze user feedback to identify recurring or systemic performance issues.
  • Contribute to the development of benchmarks and quality standards for evidence-based clinical AI.
  • Collaborate with Clinical AI Strategists, clinicians, editors, product managers, engineers, and data scientists.
  • Communicate complex findings clearly to clinical, editorial, product, and technical audiences.
  • Present evaluation findings through concise written reports, working sessions, and decision-oriented summaries.
  • Contribute to shared evaluation methods, documentation, and quality standards across the Clinical AI team.
  • Contribute to selected research, publications, and presentations related to clinical AI evaluation.

Benefits

  • Medical, dental, and vision plans
  • 401(k)
  • FSA/HSA
  • Commuter benefits
  • Tuition assistance plan
  • Vacation and sick time
  • Paid parental leave
  • Bonus eligibility
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service