Sr. AI Engineer

DRH Search•San Francisco, MA

9d•Hybrid

About The Position

We've partnered with a well-funded startup in the AI Healthtech space. They're building a healthcare advocacy platform that gives medicare patients a dedicated care advocate to navigate the healthcare system on their behalf. The role will work a hybrid schedule from their SF or Boston office, and they're open to remote for the right candidates.

Requirements

You've built AI agents or LLM-powered systems from scratch and shipped them to production — not just demos, but systems handling real interactions with real users at scale.
You have hands-on experience building evaluation pipelines for AI systems — designing metrics, capturing signals from production usage, and using that data to systematically improve agent quality.
You have experience with reinforcement learning from human feedback (RLHF), reward modeling, or other feedback-driven improvement loops for LLM-based systems.
You've monitored and debugged AI systems in production — you know what it takes to keep agents reliable when they're interacting with real people.
You're comfortable with ambiguity. The playbook doesn't exist yet, and you're excited to build it.

Nice To Haves

Experience in healthcare, health tech, or regulated industries (HIPAA, PHI handling).
Familiarity with Medicare, insurance workflows, or clinical operations.
Experience with voice AI systems — speech-to-text, text-to-speech, and real-time voice orchestration.
Background in RAG systems, vector databases, and knowledge retrieval pipelines.
Contributions to open-source AI projects or a portfolio of side projects that show your range.

Responsibilities

Launch and scale high-quality AI agents.
Build and deploy voice AI for automating patient activation and outbound calls to insurances, vendors, and providers.
Design individual agents that orchestrate end-to-end workflows such as DME delivery, prior authorizations, and care coordination.
Build copilots that assist and nudge advocates in real time.
Build the eval and reinforcement learning pipeline.
Set up the evaluation infrastructure that measures agent quality across every interaction.
Capture the right data from product usage, build the feedback loop, and implement RL pipelines so that agent performance improves continuously with real-world usage.
Create the usage-driven product improvement flywheel.
Scale AI Nurse by composing multiple agents.
Bring together voice, workflow, and copilot agents into a unified AI Nurse experience.
Ensure that as we scale, each agent reinforces the others through shared learning and a consistent improvement loop.
Ship fast and iterate with real users.
Deploy to production, monitor how agents perform with actual patients, and improve based on real conversations.
Own the tight loop from usage data to system-level improvements.
Shape the technical roadmap.
Work with the founders to decide what to build next.
Bring deep knowledge of what's possible with current AI capabilities and help us make smart bets on where the technology is heading.
Lay the foundation for scale.
Make architectural decisions that will hold up as we grow from hundreds of patients to hundreds of thousands.
Document systems, establish best practices, and build with the next engineer in mind.