Data Engineer, AI Data Platform
About The Position
Help build the world’s most advanced multimodal dataset at Microsoft AI. We are on a mission to create the largest and most advanced multimodal dataset in the world. This dataset, spanning all modalities from across the web and beyond, will power the training of the world’s most capable AI frontier models, pushing the boundaries of scale, performance, and product deployment. The AI Data Infra team at Microsoft AI is responsible for building data infrastructure to help MAI teams to generate the biggest and best training dataset. Our work involves data pipelines, Spark, Ray, Vector Databases, and all other aspects of data infra. We are looking for outstanding individuals excited about contributing to the next generation of systems that will transform the field.
Requirements
- Master's degree in computer science, Math, Software Engineering, Computer Engineering, or related field AND 4+ years' experience in business analytics, data science, software development, data modeling, or data engineering OR bachelor's degree in computer science, Math, Software Engineering, Computer Engineering, or related field AND 6+ years' experience in business analytics, data science, software development, data modeling, or data engineering OR equivalent experience.
- Experience with distributed data Platforms such as Spark, Flink or Ray.
- Proficient experience in Python and experience with SQL and Shell.
- Experience with Multimodal Data (Text, Image, Video, or Audio)
Nice To Haves
- Hands-on experience building datasets for LLM/VLM/multimodal model pre-training or post-training.
- Experience with synthetic data, including text generation, image-text synthesis, rendering/diffusion-based generation, or multimodal trajectory generation for GUI, search, file-operation, or agentic tasks.
- Familiarity with major evaluation benchmarks such as MMLU, MMBench, MM-BrowseComp, with practical experience using evaluation results and failure analysis to drive targeted data improvements.
- Experience with open file and table formats such as Lance, Iceberg, Paimon and Parquet, including schema evolution, versioning, transactions, indexing, and performance optimization for multimodal AI workloads.
Responsibilities
- Build AI Data Infrastructure and Data Engines
- Develop Intelligent Multimodal Data Processing Systems
- Build AI-Native Data Pipelines
- Build AI-Native Data Storage and Table Layers
- Drive the Data–Model–Evaluation Iteration Loop
- Partner Closely with Model and Training Teams
Benefits
- Certain roles may be eligible for benefits and other compensation.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Mid Level
