Machine Learning Engineer

Revelation Pharma LLC

6h•Remote

About The Position

The Machine Learning Engineer owns the intelligence layer that makes Evergreen's agents clinically meaningful rather than simple rule executors. This person designs, trains, validates, and monitors the ML models that power the Markov state transition logic, clinical decision support, patient risk stratification, adherence prediction, and dosage optimization. They work at the intersection of clinical domain knowledge and applied ML — building models that must be explainable to providers, safe for patients, and auditable by regulators. As the architecture matures, this role expands into the predictive analytics layer (Amazon Forecast DeepAR+ for churn risk, adherence forecasting) and the evaluation frameworks that determine whether agent recommendations meet clinical safety thresholds. This is not a research role — it is an applied ML engineering role where every model must ship, every prediction must be defensible, and every failure mode must be anticipated.

Requirements

Applied ML / Statistical Modeling (Expert): Deep experience with classification, regression, time-series forecasting, and probabilistic graphical models (Markov chains, HMMs, Bayesian networks). Must have shipped ML models to production — not just notebooks. Strong foundation in experimental design, A/B testing methodology, and statistical significance in clinical contexts.
NLP & LLM Engineering (Expert): Production experience with LLM-based systems — prompt engineering, fine-tuning, RAG architecture design, embedding models, and output evaluation. Must understand hallucination mitigation, grounding techniques, and how to constrain LLM outputs to clinically safe boundaries. Experience with Bedrock (Claude, Titan) preferred; equivalent depth with OpenAI, Vertex AI, or Azure OpenAI acceptable.
AWS ML Services (Strong): Hands-on experience with at least 3 of: SageMaker (training, endpoints, pipelines), Bedrock (agents, knowledge bases, model invocation), Amazon Forecast (DeepAR+, predictor optimization), S3 Vectors, Comprehend Medical, or HealthLake analytics. Must be comfortable building end-to-end ML pipelines on AWS.
Feature Engineering & Data Pipelines (Strong): Experience building feature stores and training data pipelines from healthcare data sources. Must understand FHIR resource structures well enough to extract clinically meaningful features from Patient, Observation, Condition, MedicationRequest, and related resources. Experience with data normalization challenges (free-text to structured, terminology mapping) is critical.
Model Evaluation & Safety (Strong): Experience designing evaluation frameworks for high-stakes predictions — clinical decision support, medical device software, or similarly regulated domains. Must understand sensitivity/specificity tradeoffs in clinical contexts, how to set appropriate confidence thresholds, and when a model should defer to human judgment.
MLOps & Production ML (Strong): Model versioning (MLflow, SageMaker Model Registry, or equivalent), automated retraining pipelines, drift detection, shadow deployments, and canary rollouts for model updates. Must have experience monitoring models in production beyond accuracy metrics — latency, cost, fairness, and distributional stability.
6+ years in ML engineering or applied data science, with at least 3 years shipping ML models to production in a healthcare, biotech, or clinical domain
Direct experience building clinical decision support, risk stratification, or patient outcome prediction models — must understand the regulatory and ethical implications of ML in healthcare
Production experience with LLM-based agent systems — must have built or significantly contributed to a system where an LLM makes consequential decisions with safety guardrails
Demonstrated ability to collaborate with clinical domain experts (physicians, pharmacists, clinical researchers) to translate domain knowledge into model design decisions
Experience working in HIPAA-regulated environments — must understand de-identification requirements, minimum necessary data access, and audit trail requirements for ML training data
Track record of building explainable models in regulated contexts — must be able to articulate why a model makes a specific prediction to both technical and clinical audiences

Nice To Haves

Advanced degree (MS or PhD) in machine learning, computational biology, biomedical informatics, statistics, or a related quantitative field
Experience with GLP-1 / obesity medicine clinical workflows, compounding pharmacy operations, or weight management program analytics
Familiarity with FDA Software as a Medical Device (SaMD) guidance and how it applies to clinical decision support systems
Experience with Comprehend Medical for clinical NLP (entity extraction, ICD/RxNorm mapping from unstructured text)
Prior work with SMART on FHIR or CDS Hooks for integrating ML-driven recommendations into clinical workflows
Publications or patents in healthcare ML, clinical NLP, or related domains (valued but not required)
Experience with causal inference methods for treatment effect estimation in observational clinical data

Responsibilities

Build the evaluation and validation framework for all agent-driven clinical recommendations — including safety guardrails, confidence thresholds, and human-in-the-loop escalation triggers for the Clinical Protocol Agent
Develop patient risk stratification models for adherence prediction, adverse event likelihood, dosage titration optimization, and churn/dropout risk using clinical, behavioral, and engagement signals
Implement and manage the predictive analytics pipeline on AWS — Amazon Forecast (DeepAR+) for time-series clinical predictions, S3 Vectors for embedding-based patient similarity and retrieval, and Bedrock for agent inference
Design and build the RAG architecture that grounds agent responses in clinical protocols, formulary data
Own model lifecycle management — training pipelines, feature stores, model versioning, A/B testing, drift detection, and retraining triggers in production
Build explainability layers for clinical recommendations
Collaborate with the clinical team to translate clinical protocols and pharmacy domain knowledge into model features, training labels, and validation criteria
Establish model monitoring and alerting — prediction quality dashboards, distribution shift detection, and automated alerts when model performance degrades below clinical safety thresholds
Work with the DevOps/AgentOps teammate to ensure all ML decisions are logged, reproducible, and auditable for regulatory review