Clinical Evaluator, AI Products (Contract)

Headway

2d • $125 - $125

About The Position

Headway is hiring contracted Clinical Evaluators to help ensure its AI-powered patient and provider products produce safe, clinically sound, high-quality outputs. Evaluators will work with Clinical and Product teams to develop and refine evaluation rubrics, assess AI outputs against those standards, and help determine whether products are ready to launch. The work requires applying clinical judgment to ambiguous cases, documenting reasoning, and collaborating with peer evaluators to reach consistent decisions.

The role requires a minimum commitment of 10 hours per week. Evaluators are paid $125 per hour and must establish or already have an LLC or S-Corp. All review and feedback must be completed within Headway-designated systems and comply with HIPAA and Headway privacy and confidentiality requirements; patient information must not be stored or maintained outside those systems.

Requirements

• Current therapist or professional counselor with an independent, unrestricted license.
• At least 3 years of direct therapy experience.
• Strong grasp of risk assessment, safety planning, and documentation standards, with the ability to identify outputs that could put a patient at risk.
• Comfort working with AI-assisted review tools, critically evaluating model outputs, and providing structured feedback.
• Ability to explain clinical reasoning clearly in writing and adjust conclusions in response to stronger peer arguments or data.
• Willingness to work through ambiguous edge cases with peers to reach consistent decisions.
• Comfort adapting to healthcare technology platforms and evolving workflows.
• Minimum commitment of 10 hours per week.
• Must set up or already have an LLC or S-Corp.

Nice To Haves

• Experience with AI chart reviews, quality audits, or clinical documentation standards.

Responsibilities

• Review AI tool outputs across Headway's patient and provider products for clinical accuracy, safety, and appropriateness.
• Apply evaluation rubrics, help improve them, use clinical judgment when a rubric falls short, and document reasoning.
• Participate in calibration sessions and inter-rater reliability rounds, working through ambiguous cases with peers to maintain consistent and defensible standards.
• Work with the Clinical Product team to turn clinical feedback into specific rubric or output changes.
• Perform additional evaluation work as AI programs grow, including labeling datasets and reviewing new product use cases before launch.
• Complete review and feedback work within Headway-designated systems and comply with HIPAA and Headway privacy and confidentiality requirements.

What This Job Offers

• Job Type: Part-time
• Career Level: Mid Level
• Education Level: No Education Listed