Machine Learning Engineer, Evaluation and Quality, Creativity Apps

Apple

1d

About The Position

The Creativity Apps team is seeking a Machine Learning Engineer focused on the quality and evaluation of machine learning models and AI features in creativity applications. This role defines how the team measures whether models and product features are ready to ship by designing evaluation strategies and metrics, building datasets and supporting infrastructure, and driving quality from model development through delivery to customers.

The engineer owns evaluation and quality for AI features across Creativity Apps and collaborates with machine learning engineering, infrastructure, product, and design teams. The role translates model and use-case requirements into evaluation methodologies, testing strategies, and quality measures that reflect creators’ experiences. It also applies machine learning, LLMs, and agentic workflows to improve evaluation, and communicates quality results and ship/no-ship recommendations to technical and non-technical audiences.

Requirements

• At least 5 years of industry experience in machine learning or software engineering, focused on ML/AI model evaluation and quality.
• Experience with evaluation methodology for machine learning models, LLMs, and AI products or features.
• Strong understanding of the machine learning development cycle, from data and training through evaluation, deployment, and monitoring.
• Solid machine learning fundamentals and LLM knowledge, including agentic workflows, and awareness of current developments in the field.
• Track record of applying LLMs or agentic workflows to machine learning evaluation and quality.
• Experience in data curation, dataset creation, and dataset quality, including iterative improvement workflows for product-specific datasets.
• Experience designing, implementing, maintaining, or optimizing machine learning evaluation infrastructure at scale.
• Experience designing evaluation metrics based on the model, product, and use cases.
• Experience driving end-to-end evaluation and quality for AI products.
• Ability to report quality and present quality data to technical and non-technical audiences.
• Strong communication skills and experience working with and driving cross-functional partners.
• Proficiency in Python and/or Swift or comparable languages.
• Bachelor’s degree in Computer Science, Data Science, Machine Learning, or a related field, or equivalent experience.

Nice To Haves

• Knowledge of the software development life cycle and software quality assurance methodologies.
• Experience in objective image and video quality evaluation and metrics.
• Knowledge of color science and/or experience with subjective image or video quality review.
• Passion for creative applications; experience as a creator in photo, video, music, or design is a plus.
• Excitement about learning new technology and applying it to practical problems.
• Familiarity with iOS and macOS.

Responsibilities

• Define evaluation strategies and metrics for machine learning models, LLMs, and AI features, tailored to the model, product, and use cases.
• Curate and create evaluation datasets, measure and improve dataset quality, and run iterative improvement loops to keep datasets representative of each product.
• Design, build, maintain, and optimize scalable evaluation infrastructure with reproducible runs, tracked baselines, and comparisons between model versions.
• Apply machine learning, LLMs, and agentic workflows to evaluation, including automated judging, test generation, and analysis.
• Drive end-to-end evaluation and quality for AI features, from model checkpoints through integrated, shipping features; triage issues and drive them to resolution.
• Report quality clearly and turn evaluation results into ship/no-ship risk assessments and recommendations for technical and non-technical audiences.
• Partner with and drive cross-functional teams, making informed judgments in ambiguous situations.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior