AIML - Machine Learning Engineer, Evaluation - Responsible AI and Safety

Apple•Cupertino, CA

About The Position

Apple is seeking an experienced engineering leader for its Responsible AI group within the Software Engineering organization. The role leads a team of machine learning engineers and data scientists evaluating, quantifying, and improving the safety and inclusivity of Apple’s generative AI features and products, including large language models, diffusion models, and multimodal systems. The position helps determine how models are evaluated, aligned, and monitored while upholding Apple’s values, privacy, and quality standards. The leader sets technical direction for operationalizing policy and risk assessments into measurable criteria, designing datasets, developing LLM- and human-in-the-loop analysis methods, and overseeing data pipelines that provide reliable, scalable insights for product ship/no-ship decisions. The role remains hands-on before and after launch by analyzing evaluation results, investigating failures, and using human and automated feedback to improve safety and user trust. It also involves mentoring and developing the team, fostering an inclusive culture of technical excellence, advising senior leadership on safety, resolving roadblocks, and managing cross-functional stakeholders. The work relies on applied data science, scientific investigation and interpretation, requirements negotiation, defending safety metrics, and reporting results to stakeholders and decision-makers.

Requirements

MS or PhD in Computer Science, Linguistics, Cognitive Science, HCI, Psychology, Mathematics, Physics, or a similar science or technology field with a strong basis in scientific data collection and analysis, plus at least 4 years of relevant work experience; or BA/BS with 8+ years of relevant work experience.
At least 2 years of direct engineering-team management experience.
Experience gathering and analyzing varied language, image, voice, video, and/or multimodal data, including human- and LLM-generated data.
Experience with LLM judges and human-in-the-loop techniques, including designing human annotation projects, writing guidelines, and handling highly multilabeled, nuanced, and potentially conflicting data.
Proficiency in data science, machine learning, analytics, programming with Python and Pandas, and AI-assisted coding tools.
Experience with one or more plotting and visualization libraries.
Ability to collaborate cross-functionally, prioritize competing project needs, set and maintain schedules for project milestones, and communicate with team members and internal and external stakeholders at all levels.
A rigorous approach to metric development, interpretation of experiments and evaluations, and presentation to executives.
Comfort leading a team working with sensitive or potentially offensive content.

Nice To Haves

Experience working in the Responsible AI space.
Passion for building innovative, high-impact products and teams drawing on interdisciplinary skills.
Curiosity about fairness and bias in generative AI systems and a desire to help make the technology more equitable.
A Social Science or Humanities background, or another demonstrable interest in socio-technical issues.
Prior scientific research and publication experience.
A proven track record of contributing to diverse teams in a collaborative environment.
Experience working with multimodal models, agentic systems, and/or diverse languages and locales.
Experience with AI compliance and regulatory needs, particularly child safety, clinical or health applications, or companionship chatbots.
Experience conducting large-scale benchmarking across models and capabilities.

Responsibilities

Build, mentor, and develop a high-performing team of machine learning engineers and data scientists.
Set technical direction for Responsible AI evaluation efforts for generative AI products.
Translate policy and risk assessments into measurable criteria.
Design datasets and develop LLM- and human-in-the-loop methods for data analysis.
Oversee implementation of data pipelines that produce reliable, scalable insights for ship/no-ship decisions.
Analyze evaluation results before and after product launch.
Conduct failure investigations and use human and automated feedback to improve feature safety and user trust.
Collaborate with cross-functional partners to reduce unintended consequences and improve feature capabilities and user experience.
Foster an inclusive culture of technical excellence, empower senior leadership with safety recommendations, and remove roadblocks for the team.
Manage cross-functional stakeholder relationships, negotiate requirements, defend safety metrics, and report findings to stakeholders and decision-makers.