Machine Learning Research Engineer, Pretraining Data — Foundation Models

Apple • Cupertino, CA

18h

About The Position

Apple’s Foundation Models team is building large language models that power intelligent experiences across Apple products. The Pretraining Data team seeks a Research Engineer to advance pretraining data research and engineering, developing algorithms and scalable systems for data curation, quality assessment, and optimization. The work connects machine learning research with large-scale systems engineering and contributes to improving foundation-model capabilities in knowledge, reasoning, coding, and multilingual understanding.

The role owns technically challenging projects across research, algorithm development, experimentation, and production engineering. It investigates ways to improve training-data quality, diversity, and efficiency; builds infrastructure for massive datasets; and collaborates with model-training researchers to understand how data affects model capabilities and to improve pretraining strategies.

Requirements

• BS, MS, or PhD in Computer Science, Machine Learning, or a related technical field, or equivalent practical experience.
• Strong programming skills in Python and proficiency in developing production-quality software.
• Experience with large-scale data processing, distributed computing, or machine-learning systems.
• Solid understanding of machine-learning fundamentals and modern deep-learning architectures.
• Strong analytical and problem-solving skills, including the ability to independently investigate complex technical challenges.
• Experience with LLM pretraining, training-data research, or foundation-model development.
• Experience designing and implementing scalable data-processing pipelines using Spark, Ray, or similar distributed-computing frameworks.
• Hands-on experience with PyTorch, model training, and experimental evaluation.
• Familiarity with data filtering, deduplication, data-mixture optimization, and learned data-quality signals.
• Understanding of scaling laws, training dynamics, and the relationship between data and model performance.
• Experience with synthetic-data generation, data selection, or model-based data evaluation.
• Demonstrated ability to translate research ideas into scalable production systems.
• Publications or meaningful contributions to machine-learning research or open-source projects.

Responsibilities

• Research, design, and implement algorithms for large-scale pretraining data curation, including quality filtering, deduplication, data selection, and mixture optimization.
• Build and optimize scalable data-processing systems capable of handling trillions of tokens across diverse data sources.
• Develop machine-learning approaches for data-quality assessment, synthetic-data generation, and data-efficiency optimization.
• Collaborate with foundation-model researchers to design and execute training experiments, analyze model performance, and identify opportunities to improve model capabilities.
• Investigate relationships between training-data characteristics, model scaling, training dynamics, and downstream evaluation results.
• Translate research prototypes into efficient, reliable, production-ready data systems.
• Stay current with advances in foundation-model research and apply emerging techniques to practical data challenges.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level