Member of Technical Staff (ML)

Solstice•New York City, NY

8d•Onsite

About The Position

Solstice builds AI software that helps life sciences teams create, refine, and review pharmaceutical marketing content. The company has worked with leading life sciences manufacturers and more than 50 pharma brands. As a Member of Technical Staff in Machine Learning, you will improve the models, representations, and data powering Solstice's AI agents. Your work will help agents find supporting evidence, use tools accurately, and incorporate reviewer feedback into better outputs. You will investigate failures across complete agent runs, form testable hypotheses, implement changes, and assess whether those changes improve customer tasks. Potential approaches include improving training examples, retrieval or ranking, models or context strategies, and targeted fine-tuning. You will evaluate quality, reliability, latency, and cost before production deployment. You will work directly with Applied AI engineers, who own complete features and workflows, and Fullstack AI engineers, who build the shared runtime, APIs, and product experiences. You will build prototypes and production components together, with primary responsibility for model and data improvements and evidence that they generalize.

Requirements

Strong machine-learning fundamentals and hands-on implementation skills, including reasoning about learning objectives, optimization, regularization, data quality, and generalization.
Strong Python skills and hands-on experience building or improving LLM systems, retrieval systems, or agents.
Ability to inspect tool-call traces, reproduce failures, and implement experiments or production changes to investigate them.
Evidence of taking a data or model improvement into production and understanding what happened after deployment.
Sound experimental judgment, including explicit hypotheses, credible comparisons, meaningful baselines, and care with leakage, noisy labels, and misleading aggregate metrics.
Depth in at least one relevant area, such as retrieval and ranking, representation learning, model adaptation, or learning from human feedback, and ability to apply it to multi-step agent tasks.
Judgment to determine whether failures call for changes to data or models, context, or tool interfaces, and to design comparisons that distinguish those explanations.
Practical software engineering skills for reproducible experiments, tested components, production integration, and debugging.
Clear communication, personal ownership, ability to explain tradeoffs and work with product and engineering partners, and willingness to follow through when results challenge assumptions.

Nice To Haves

Experience with scientific documents, human review feedback, or regulated enterprise products.
Healthcare or pharma experience.

Responsibilities

Diagnose failures in evidence selection, tool choice and arguments, and revisions based on feedback; work with Applied AI to distinguish model or data limitations from missing context, tool failures, or workflow design problems.
Develop and compare retrieval, reranking, and claim-to-evidence matching approaches across scientific documents and approved content, measuring whether evidence supports claims and helps agents complete tasks.
Build curated datasets from successful and failed agent runs, corrections, and targeted examples; track provenance, resolve ambiguous labels, and keep held-out evaluation cases separate from training and tuning.
Compare model choices, representations, context strategies, and learning methods using credible baselines and controlled experiments; use targeted fine-tuning or other model adaptation when justified by data and results.
Evaluate complete-task outcomes, intermediate tool behavior, resulting artifacts, repeated-run reliability, and generalization to unfamiliar documents and task variants.
Calibrate automated judgments against human review and track quality alongside latency and cost.
Integrate improvements into production Python services and agent workflows.
Use versioned data and models, staged releases, quality monitoring, and rollback criteria; investigate whether offline gains hold up in use.