AIML - ML Engineer, Safety Human Evaluation

Apple • Cambridge, MA

15h

About The Position

As a member of Apple’s HCMI/Responsible AI group, the ML Engineer will work on generative models that power new Apple features and contribute to longer-term generative AI research. The role focuses on assessing, quantifying, and improving the safety and inclusivity of Apple’s generative-AI features and products, including systems based on large language models, diffusion models, and multimodal models. Areas of interest include Responsible AI, safety, fairness, robustness, explainability, and model uncertainty.

The work combines applied data science, scientific investigation and interpretation, cross-functional collaboration, and reporting metrics to stakeholders and decision-makers. It includes creating and curating data and annotations, evaluating safety concerns, and helping improve user-facing Apple Intelligence products. The role also involves working with highly sensitive material, including offensive and controversial content.

Requirements

• MS or PhD in Computer Science, Linguistics, Cognitive Science, HCI, Psychology, Mathematics, Physics, or a similar science or technology field with a strong basis in scientific data collection and analysis, plus at least 4 years of relevant work experience; or a BA/BS with 8 or more years of relevant work experience.
• Experience gathering and analyzing language, image, and/or multimodal data, including LLM-generated data.
• Strong experience designing human annotation projects, writing guidelines, and handling highly multi-labeled, nuanced, and often conflicting data.
• Proficiency in data science, machine learning, analytics, and programming with Python and Pandas.
• Strong experience with one or more plotting and visualization libraries.
• Ability to collaborate with team members to prioritize competing projects, establish and maintain schedules for milestones and project completion, and communicate with team members at all levels and external stakeholders.
• Strong skills in rigorous model-quality metric development, interpretation of experiments and evaluations, and presentation to executives.

Nice To Haves

• Experience working in the Responsible AI space.
• Curiosity about fairness and bias in generative AI systems and a strong desire to help make the technology more equitable.
• Prior scientific research and publication experience.
• Experience working with generative models for evaluation and/or product development, and current knowledge of common challenges and failures.
• A proven track record of contributing to diverse teams in a collaborative environment.
• A passion for building outstanding and innovative products.

Responsibilities

• Develop metrics to evaluate safety and fairness risks in generative models and generative-AI features.
• Design datasets, identify data needs, and expand data coverage using human and synthetic generation methods.
• Develop sampling strategies and combine human annotation with automated grading to produce high-quality, high-confidence insights at scale and pace.
• Use and implement data pipelines, and collaborate across functions to execute end-to-end safety evaluations.
• Translate project findings into recommendations for product engineering teams and safety policy development.
• Develop machine-learning-based enhancements to red teaming, model evaluation, and other processes to improve Apple Intelligence’s user-facing products.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior