Staff ML Platform Engineer (MLOps)

FutureFit AI•New York, NY

8d•$172,000 - $215,000•Remote

About The Position

FutureFit AI is seeking a Staff ML Platform Engineer (MLOps) to build and own the platform that powers our ML and LLM-powered products. The ML footprint has grown rapidly, and this role is crucial for developing the underlying infrastructure. You will be responsible for the end-to-end lifecycle of models, including their building, deployment, evaluation, and serving. This includes managing compute and environments, optimizing LLM calls for cost, and ensuring robust monitoring to detect issues quickly. This is a hands-on role that involves both building new platform components and operating existing systems, requiring the ability to debug and fix issues when models regress or performance degrades.

Requirements

Staff-level, hands-on experience in MLOps, ML platform, or ML infrastructure.
Experience establishing MLOps practices end-to-end: CI/CD for models, experiment tracking, model registries, deployment workflows, and monitoring.
Production experience with LLM-based systems: serving, prompt and response evaluation, routing across models and providers, and managing cost and latency tradeoffs.
Experience operating models in both batch and real-time serving contexts.
Hands-on experience with compute provisioning and environment management: containers, reproducible training and serving environments, and keeping frameworks and packages current.
Experience running controlled model experiments in production: A/B tests, shadow or canary deploys, holdouts, and setting success criteria.
Depth in observability and traceability for production ML: drift and regression detection, alerting, lineage, and the ability to trace predictions.
Hands-on operational experience: carrying the pager or equivalent, debugging production ML incidents, and fixing systems not originally built.
Comfort with data engineering for the platform: pipelines, feature computation and storage, and consistency between training and serving.
A track record of diagnosing problems in complex, fast-grown systems and materially improving them.
Strong systems design ability: translating product needs into durable architecture and building it.

Nice To Haves

Interest in growing into model development.
Feature store experience.
Experience evaluating AI/ML observability or LLM evaluation vendors.
Background in mission-driven, workforce, or government-adjacent data environments.
Comfort mentoring a small data and engineering team.

Responsibilities

Evaluate current pipelines, data architecture, and ML workflows to create a prioritized plan for platform improvements.
Own compute provisioning and environment management, ensuring reproducible training and serving environments, and keeping frameworks and packages up-to-date.
Build and manage the infrastructure for LLM features, including smart routing for cost optimization and prompt/response evaluation to maintain quality.
Implement a disciplined approach to A/B testing models through shadow deploys, canaries, holdouts, and pre-defined success criteria.
Develop and maintain observability and traceability for production ML systems, including model and data monitoring, alerting, regression detection, and lineage.
Operate running systems, debug production ML incidents, and fix systems, including those not originally built by the individual.
Own the computation, storage, and serving of features, ensuring consistency between training and inference, and performing necessary data engineering tasks.
Establish deployment and monitoring standards for the ML team to foster shared ownership.