Machine Learning Operations (MLOps) Engineer

Panera Bread

9d•$127,461 - $155,477•Hybrid

About The Position

The MLOps Engineer builds and operates the platform and pipelines that move machine learning and AI models from development into reliable production use. The role owns model deployment, automation, monitoring, and lifecycle management, and serves as the ongoing owner of enterprise sales forecasting models. The engineer works within models to diagnose accuracy degradation and retune features, algorithms, and model logic so forecasts continue to deliver measurable business value. The role partners with Data Science, Data Engineering, Security, Finance, Operations, and Supply Chain teams, and is accountable for the operational health of live models. The position includes deploying and managing models on Vertex AI; operationalizing LLM and generative AI workloads; creating monitoring, retraining, registry, versioning, and lineage practices; managing feature pipelines; optimizing compute and inference costs; and defining infrastructure-as-code standards. It also supports model governance, security and privacy requirements, production troubleshooting, architecture reviews, and mentoring on production standards. This is a hybrid position with onsite presence required in Newton, Massachusetts, or St. Louis, Missouri. Competitive annual pay is $127,461 to $155,477. Actual pay depends on factors including relevant experience, job-related knowledge, skills, and geographical location.

Requirements

Bachelor's degree in Computer Science, Engineering, Data Science, Analytics, or a related field.
At least 3 years of experience in MLOps, Machine Learning Engineering, Data Engineering, or DevOps supporting production ML workloads.
Demonstrated experience deploying and operating production machine learning models and maintaining and improving models after launch.
Ability to independently modify model features, algorithms, and hyperparameters, rather than only deploy models built by others.
Experience owning and improving production time series or demand forecasting models, including diagnosing and correcting accuracy degradation.
Working knowledge of forecasting techniques and accuracy measures such as WMAPE, and ability to explain tradeoffs between model options.
Proficiency with common machine learning and statistical libraries used for model development and evaluation.
Strong Python proficiency, including packaging, testing, and writing production-quality code.
Hands-on experience with Google Cloud Platform, particularly Vertex AI and BigQuery.
Experience building CI/CD pipelines and applying software engineering practices to ML workflows.
Experience with containerization and orchestration such as Docker, Kubernetes, or similar tools.
Experience with infrastructure-as-code tooling such as Terraform or similar.
Working knowledge of workflow orchestration tools such as Airflow, Cloud Composer, Vertex AI Pipelines, or similar.
Strong SQL proficiency and understanding of data pipeline and warehouse concepts.
Experience implementing model monitoring, drift detection, and automated alerting.
Understanding of model evaluation metrics and ability to reason about model performance in production.
Structured problem-solving skills and persistence in driving production issues to root cause.
Excellent written and verbal communication skills, including ability to explain operational risk to technical and non-technical stakeholders.

Nice To Haves

Hands-on experience operationalizing LLMs, including Gemini via Vertex AI or in-warehouse invocation from BigQuery.
Experience with LLM operational concerns such as prompt versioning, output evaluation, guardrails, and token cost management.
Experience with sales or demand forecasting in a multi-unit retail or restaurant environment, including café- or store-level granularity.
Experience benchmarking competing forecasting approaches and recommending models based on measured accuracy and business impact.
Experience with feature store implementation and management.
Experience managing models delivered by external vendors or systems integrators through production handover.
Familiarity with responsible AI practices, model risk management, or AI governance frameworks.
Restaurant, retail, or multi-unit operations data experience.

Responsibilities

Design, build, and maintain CI/CD pipelines for model training, validation, deployment, and rollback.
Productionize models developed by data scientists and vendor partners as scalable, maintainable, and reproducible pipelines.
Own the ongoing maintenance and performance of enterprise sales forecasting models, including accuracy tracking, recalibration, and continuous improvement.
Diagnose forecast accuracy degradation and improve results through feature engineering, algorithm selection, hyperparameter tuning, and changes to model logic.
Evaluate forecast performance using measures such as WMAPE and report results and improvement trends to business stakeholders.
Translate business context—including promotions, seasonality, café openings and closures, and operational disruptions—into model features and adjustments.
Partner with Finance, Operations, and Supply Chain to understand forecast use and ensure model changes deliver measurable business benefit.
Deploy and manage models on Vertex AI, including training jobs, endpoints, batch prediction, and pipeline orchestration.
Operationalize LLM and generative AI workloads, including in-warehouse model invocation from BigQuery, prompt and model version management, and output validation.
Implement automated monitoring and alerting for model accuracy, drift, data quality, latency, and failures.
Build automated retraining workflows with promotion criteria and approval gates before new model versions reach production.
Establish model registry, versioning, and lineage practices to trace production predictions to model versions, code commits, and training datasets.
Manage feature pipelines and maintain consistency between training and serving data to prevent training-serving skew.
Monitor and optimize compute and inference costs, including model selection, batch versus real-time tradeoffs, and capacity utilization.
Define and enforce infrastructure-as-code standards for ML environments and environment promotion from development through production.
Partner with Security and Privacy teams to ensure models and training data meet data classification, access control, and retention requirements.
Support model governance and audit requirements by documenting model behavior, validation results, and approval history.
Troubleshoot production model and pipeline failures, lead root cause analysis, and implement preventive controls.
Collaborate with Data Engineering to define upstream data requirements and ensure reliability of model input pipelines.
Contribute to architecture reviews and present ML platform designs and operational readiness for approval.
Mentor engineers and data scientists on production standards, deployment practices, and operational discipline.