Multimodal Infrastructure

Microsoft•,

7d•$142,800 - $331,200•Hybrid

About The Position

Microsoft AI is seeking a Member of Technical Staff, Multimodal Infrastructure to contribute to the development of its personalized AI assistant, Copilot. The role involves building world-class consumer experiences and products in a fast-paced environment, with a focus on developing AI models that power innovative products. Responsibilities span model architecture, data curation, training and inference infrastructures, evaluation protocols, alignment, and reinforcement learning from human feedback (RLHF). The organization is dedicated to building foundational models for responsible and efficient artificial general intelligence, requiring large compute capacity. The Member of Technical Staff will be responsible for building large-scale infrastructures to support the full cycle of multimodal generative model development. This includes working closely with research scientists and product engineers on multimodal data processing, model training, inference, and serving tasks. The role also involves contributing best practices, driving architectural changes, and influencing the roadmap of relevant software and hardware components, directly impacting business goals and AI innovation. Microsoft AI is responsible for Copilot, Bing, Edge, and generative AI research, aiming to shape the future of personal computing. Microsoft's mission is to empower every person and every organization to achieve more, fostering a culture of growth mindset, innovation, and collaboration built on values of respect, integrity, and accountability.

Requirements

Bachelor's Degree in Computer Science, or related technical discipline AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience

Nice To Haves

Bachelor's Degree in Computer Science or related technical field AND 10+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR Master's Degree in Computer Science or related technical field AND 8+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience
Experience in multi-modal data processing: Strong proficiency in distributed data processing infra (resource utilization management, fault tolerance, ray & spark) and CPU/GPU batch processing optimizations
Experience with state-of-art model inference and serving frameworks
Experience with image/video/audio data processing
Experience with common data formats for efficient I/O
Experience in multi-modal pretraining and post-training: Strong proficiency in deep learning frameworks such as PyTorch, Megatron and Deepspeed
Knowledge of auto-regressive and diffusion transformer models
Experience with distributed training techniques such as data parallelism, model parallelism, and pipeline parallelism
Proven experiences in at least one of the following areas: image/video generation and editing; efficient architectures (e.g., MoE, window attention); efficient model design; or reinforcement learning training methods (e.g., RLHF, DPO, GRPO)
Experience in multi-modal inference and serving: Strong proficiency in serving frameworks such as vLLM, TensorRT-LLM, SGLang, xDiT, Cache-DiT etc.
Knowledge of distillation techniques such as Progressive Distillation, DMD, Self forcing etc.
Knowledge of quantization and compression techniques like AWQ, GPTQ, and FP8 for multi-modal pipelines
Experience in distributed inference scaling across multi-node clusters using Ray Serve and Triton
Experience in leading technical projects and supporting architectural decisions with data

Responsibilities

Design, develop and maintain large-scale multimodal data processing pipelines.
Design, develop and maintain large-scale multimodal model pretraining and post-training frameworks.
Design, develop and maintain large-scale multimodal model inference and serving frameworks.
Work with research scientists and product engineers to solve infra-related problems.
Find a path to get things done despite roadblocks to get your work into the hands of users quickly and iteratively.
Enjoy working in a fast-paced, design-driven, product development cycle.
Embody our Culture and Values.