Machine Learning Engineer, AI/ML Model Quality

Apple • San Diego, CA

1d

About The Position

The Machine Learning Engineer, AI/ML Model Quality will build and maintain intelligent systems, validation frameworks, synthetic data generation models, and monitoring pipelines that support data quality for machine learning and generative AI across Wallet, Payments, and Commerce. The role owns the health of training and validation datasets, defines and analyzes observability metrics, builds models to calibrate and measure robustness, and leads telemetry analysis across GenAI workflows. The work helps ensure models are trained, evaluated, and deployed using trustworthy data, and supports financial features used by hundreds of millions of people.

This role sits at the intersection of machine learning, statistical rigor, and production systems. The engineer collaborates closely with Data Engineering, Privacy, and Legal teams, identifying quality signals and translating them into actionable insights. Responsibilities span validation frameworks, observability metrics, telemetry analysis, and the monitoring of model health and data quality in production.

Requirements

• Bachelor's degree with exceptional hands-on experience in ML/AI model quality or applied research, or an M.S. or Ph.D. in Machine Learning, Computer Science, Data Science, Statistics, Mathematics, Engineering, or a related quantitative field is strongly preferred.
• At least 3 years of experience defining and tracking ML model health metrics in production, including model performance monitoring, feature drift detection, and observability instrumentation.
• At least 3 years of experience in machine learning, data science, or a closely related analytical role, focused on data quality, model evaluation, or ML observability in production environments.
• Proficiency in Python, including Pandas, NumPy, and Scikit-learn; common ML frameworks such as PyTorch; and SQL for complex data analysis, metric creation, and validation.
• Experience querying and analyzing large-scale datasets using distributed computing frameworks such as PySpark, Spark, or distributed SQL.
• Solid understanding of statistical methods, including hypothesis testing, distribution analysis, data drift detection, and statistical process control.
• Experience with GenAI or LLM systems, including common quality failure modes, output evaluation approaches, and telemetry instrumentation.
• Strong communication skills and ability to explain complex data quality findings and model health risks clearly to engineering and non-technical stakeholders.

Nice To Haves

• Experience with data visualization and dashboarding tools such as Tableau, Apache Superset, or Databricks.
• Familiarity with LLM evaluation frameworks such as LangSmith or techniques such as LLM-as-a-judge.
• Experience with Bayesian or causal graph-based approaches to synthetic data generation.
• Familiarity with confidence calibration techniques and uncertainty quantification.
• Experience with ML monitoring or observability platforms such as MLflow, Weights & Biases, or equivalent.
• Experience working with privacy-constrained data or under regulatory compliance frameworks such as GDPR or DMA.
• Background in financial services, fintech, or consumer payment products.

Responsibilities

• Build and maintain intelligent systems, data validation frameworks, synthetic data generation models, and monitoring pipelines.
• Own the health of training and validation datasets supporting ML and GenAI features across Wallet, Payments, and Commerce.
• Define and analyze observability metrics for data and model health.
• Build models to calibrate and measure robustness.
• Lead telemetry analysis across generative AI workflows.
• Monitor model performance, feature drift, and data quality in production.
• Translate data quality findings and model health risks into actionable insights for engineering and non-technical stakeholders.
• Collaborate with Data Engineering, Privacy, and Legal teams.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level