ML Engineer

Zus Health•Boston, MA

2d•$190,000 - $250,000•Hybrid

About The Position

Zus Health is hiring a Staff/Principal AI/ML Engineer to define how LLMs and AI agents power its products and underlying AI platform. This hands-on senior role owns architecture, tooling, and roadmap, combining production engineering and rapid prototyping with technical leadership and mentoring. The engineer will work with solution engineers, product managers, sales, marketing, providers, and other engineers to turn fragmented, multimodal clinical data into structured, trustworthy insight safely and at scale. The role includes architecting shared LLM infrastructure; building agentic systems and retrieval over clinical data; establishing rigorous model evaluation; adapting models through fine-tuning and distillation; and operating AI/ML systems in production. The engineer will also prototype with users, set technical standards, lead design reviews, document decisions, mentor colleagues, and help the company keep pace with developments in AI.

Requirements

At least 7 years building and shipping AI/ML systems, including at least 2 years putting LLM-powered products into production for real users.
Hands-on expertise in RAG, agents and tool use, structured outputs, prompt and context engineering, and trade-offs between frontier APIs and open-weight models.
Track record building ML infrastructure used by others, including serving, pipelines, model and prompt versioning, CI/CD, and monitoring.
Experience designing evaluation datasets and metrics and applying a rigorous approach to shipping models.
Strong Python and software engineering fundamentals, with fluency in PyTorch and the Hugging Face ecosystem.
Production experience on a major cloud platform, with containers and infrastructure as code; AWS is preferred.
Sound judgment about safety, privacy, and reliability when models handle sensitive data.
Clear writing and communication skills, including the ability to explain model behavior and risk to clinicians, product leaders, and executives.
Bachelor's degree in Computer Science, Statistics, or a related field, or equivalent experience.
Located in the Boston metro and willing to collaborate in person on a hybrid schedule, or located remotely in EST/CST and willing to travel occasionally for in-person collaboration.

Nice To Haves

Healthcare or clinical NLP experience, especially with FHIR, C-CDA, clinical terminologies, or HIPAA-regulated data.
Experience fine-tuning, distilling, or preference-optimizing LLMs at meaningful scale.
Experience building MCP servers or production multi-agent systems.
Inference optimization experience, including quantization, speculative decoding, batching, or GPU cost management.
Experience designing and executing pre-training of transformer models.
Open-source contributions, publications, or talks in ML, NLP, or LLM systems.
Experience as the first or founding ML engineer at a growth-stage company.
Master's or PhD in a quantitative field.

Responsibilities

Design the shared LLM platform, including model gateway and routing, prompt and version management, caching, cost and latency controls, and PHI-safe guardrails.
Build and ship tool-using agents that analyze longitudinal patient records, summarize histories, reconcile medications, surface care gaps, and answer clinician questions with cited evidence.
Build RAG and hybrid search over FHIR resources and unstructured clinical notes, including chunking, embedding, vector and graph retrieval, reranking, and source-grounded answers.
Establish offline and online evaluation systems, including clinical-reviewer golden datasets, LLM-as-judge pipelines, CI regression suites, and production monitoring for hallucination, drift, and safety.
Fine-tune and distill models for clinical extraction, coding, and normalization tasks when specialized models are appropriate.
Build and operate production AI/ML systems, including model and prompt CI/CD, inference serving, observability, and human-in-the-loop feedback loops.
Work with providers, product managers, and engineers to identify high-impact problems, ship prototypes quickly, and iterate based on real-world usage.
Lead design reviews, create technical documentation and standards, mentor engineers, and guide technical direction.