Lead Software Platform Engineer, MLOps

TetraScience • Cambridge, MA

5d • $200,000 - $270,000 • Remote

About The Position

TetraScience is seeking a Lead Software Platform Engineer to work at the intersection of distributed systems and MLOps. This role involves owning and scaling the company's AI and data infrastructure, which serves as a product for customers and a foundation for other engineering teams. The engineer will architect cloud-based services and MLOps infrastructure to enable production-grade AI/ML workflows, collaborating with Applied AI engineers, data engineers, and platform teams. The position requires acting as the technical design authority for how models, LLMs, and agents operate in production, covering the entire model and prompt lifecycle, evaluation and observability, security, and cost/latency controls for economically viable AI at scale. This is a high-impact role for an engineer experienced in shipping AI/ML infrastructure as a multi-tenant product, with responsibilities including architecting the AI/ML platform's service and API surface, managing the end-to-end model and prompt lifecycle, designing the inference substrate for real-time and batch workloads, integrating AI models and LLMs using architectures like RAG, and architecting the agentic layer. The role also involves designing security into the AI platform, building evaluation and quality infrastructure, establishing observability, designing for reproducibility in regulated environments, contributing to infrastructure-as-code and deployment automation, owning production readiness, acting as an SME and design authority, and setting technical direction on emerging AI infrastructure.

Requirements

• 10+ years of professional experience in software engineering and infrastructure engineering, with a proven track record of designing, building, and scaling distributed, cloud-native systems in production.
• Demonstrated experience as a technical leader or architect, accountable for the key decisions on system design, scalability, performance, and cost optimization.
• Experience designing security into a multi-tenant platform, including authorization boundaries between tenants and handling of sensitive data such as PII and PHI, with awareness of LLM-specific risks like prompt injection and tool abuse.
• Extensive experience building and maintaining AI/ML infrastructure in production, including model deployment and lifecycle management, delivered as a multi-tenant product with external users rather than internal tooling for a single team. Candidates whose experience is limited to building pipelines for their own team will not be a fit.
• Deep, hands-on experience taking LLM-based systems to production, including RAG architecture, retrieval and embedding design, prompt and model versioning, and tool or function calling. Not just prototyping with an SDK. We are looking for someone who has operated these systems under real traffic, real latency budgets, and real failure modes.
• Expert-level coding skills in TypeScript and Python building robust APIs and backend services, with the judgment to critically evaluate AI-generated code for correctness, security, and architectural fit.
• Production-level experience with a model registry and serving stack, ideally Databricks MLflow, including model registration, versioning, asset bundles, and serving workflows.
• Experience treating AI evaluation as a release gate rather than post-hoc reporting, including eval harnesses, regression gates, and drift or quality monitoring for non-deterministic systems.
• Proficiency in API-first design, including REST and OpenAPI specifications, designing APIs that are scalable, secure, versioned, and extensible.
• Solid working knowledge of AWS and containerized workloads (e.g., Docker), and familiarity with infrastructure-as-code such as CloudFormation or CDK, with the ability to contribute to CI/CD pipelines and deployment automation.
• Experience defining observability and SLI/SLO/SLA practice for production systems, including monitoring, alerting, and distributed tracing.
• Ability to articulate ideas clearly to customers and cross-functional teams, influence technical direction on teams you do not manage, and mentor other engineers.

Nice To Haves

• Familiarity with emerging LLM frameworks for advanced prompt orchestration and programmatic LLM pipelines.
• Experience running agentic or multi-step orchestration in production, and with MCP as an integration surface for tooling and non-human identities.
• Understanding of LLM cost monitoring, latency optimization, and usage analytics in production environments, including per-tenant cost attribution for tokens, GPU, and inference.
• Experience with multimodal model inputs, including image and instrument data, and the throughput and cost implications of serving them at scale.
• Experience with fine-tuning, distillation, or model optimization such as quantization, batching, or KV-cache strategy, to improve latency and cost.
• Experience delivering ML or AI systems in a regulated or validated environment (GxP, 21 CFR Part 11, SOC 2), including computer system validation and audit readiness.
• Background in scientific, life sciences, or laboratory data domains.

Responsibilities

• Own the technical architecture of the AI/ML platform: the service and API surface our customers use to run models and agents against their own scientific data, and that Applied AI and data engineering teams build against internally.
• Own the end-to-end model and prompt lifecycle across Databricks MLflow and AWS Bedrock, including registration, versioning, asset bundles, staged promotion, rollback, and multi-model serving.
• Design the inference substrate for both real-time and batch AI workloads, including routing, batching, caching, concurrency control, GPU and accelerator capacity planning, handling of large binary inputs such as instrument images, and graceful degradation under load.
• Integrate AI models and large language models (LLMs) into production systems using architectures like retrieval-augmented generation (RAG), and architect the agentic layer above them: tool and function calling, MCP-based tooling, and agent runtimes, deciding what belongs in the platform versus in the applications built on top of it.
• Design security into the AI platform rather than leaving it to the applications above it, partnering with our security team on guardrails, prompt-injection and tool-abuse defenses, PII and PHI handling, and hard tenant data boundaries across prompts, retrieval, and tool calls.
• Build the evaluation and quality infrastructure that makes AI shippable: offline and online eval harnesses, golden datasets, regression gates in CI, A/B and shadow deployment, and drift and hallucination detection in production.
• Establish observability for the AI platform, including monitoring, alerting, logging, and distributed tracing, and set the SLI, SLO, and SLA model for systems whose outputs are probabilistic.
• Design for reproducibility and lineage required in a validated pharma environment, with versioned data, code, prompts, and model artifacts, and an audit trail that can withstand customer and regulatory scrutiny.
• Contribute to the infrastructure-as-code and deployment automation for the AI platform (CloudFormation, AWS CDK), partnering with the team that owns production deployments to support multi-tenant infrastructure, online upgrades, and on-demand compute allocation.
• Own production readiness for the AI platform with Applied AI engineers, data engineers, and platform teams: the performance, reliability, and cost-efficiency of models in production, plus incident response and runbooks.
• Act as SME and design authority across product and engineering. Lead design reviews, write the reference architectures and technical documentation others follow, and mentor senior and mid-level engineers on distributed systems and AI engineering practice.
• Set technical direction on emerging AI infrastructure: evaluate new frameworks, serving runtimes, model providers, and data types, and make clear build-versus-buy decisions grounded in cost, risk, and scalability.

Benefits

• 100% employer-paid benefits for all eligible employees and immediate family members
• Unlimited paid time off (PTO)
• 401K
• Flexible working arrangements - Remote work
• Company paid Life Insurance, LTD/STD
• A culture of continuous improvement where you can grow your career and get coaching

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed