Principal Distributed Systems Software Engineer, Collectives

Advanced Micro Devices, Inc•San Jose, CA

1d•Onsite

About The Position

As a core member of AMD’s Collectives and Network Performance team, the Principal Distributed Systems Software Engineer will help define and deliver AI pre-training and distributed inference performance on AMD GPUs. The role focuses on optimizing collective communication libraries, scale-up and scale-out networking software, and training and inference performance across multi-GPU and multi-node systems. The engineer will collaborate with internal hardware, GPU library, compiler, runtime, and ROCm teams, as well as open-source maintainers, to integrate optimizations. The role applies advanced engineering principles to improve performance across multiple generations of AMD GPUs and a wide range of AI models. It also involves technical leadership, defining goals and development scope, guiding teams, and owning initiatives independently.

Requirements

Advanced technical and analytical skills in Linux environments.
Deep expertise in C++.
Deep knowledge of network, NIC, and GPU hardware architecture.
Hands-on experience mapping model architectures to low-level software and hardware.
Strong grasp of current trends in distributed training and inference at scale.
Senior-level industry experience in technical software development focused on performance engineering, communication libraries, networking, and GPU/distributed systems.
Master’s and/or PhD degree in Computer Science, Computer Engineering, Electrical Engineering, or a related field.

Nice To Haves

Deep expertise designing and optimizing collective communication libraries such as RCCL, NCCL, or MPI, and thorough understanding of NIC, RDMA, scale-up, and scale-out network architectures and their impact on model performance.
Strong technical expertise and proven experience in performance analysis, projection, and bottleneck identification across the stack, from model to hardware.
Proven ability to integrate optimized communication into ML frameworks such as PyTorch, JAX, vLLM, and SGLang, focusing on throughput, scalability, and efficient execution for training and inference.
Advanced proficiency in C++ and Python, with deep experience in performance tuning, debugging, and robust test design for reliable, maintainable, high-performance codebases.
Broad and in-depth experience with large-scale, heterogeneous compute environments and optimizing AI workloads for performance, efficiency, and resource utilization across clusters.
Demonstrated ability to lead cross-team initiatives, mentor and coach engineers, and communicate effectively in writing and presentations, internally and externally.

Responsibilities

Define goals, scope, and the roadmap for a major area of AMD’s collectives and network optimizations, aligning them with hardware, framework, and ROCm roadmaps.
Lead the design and development of collective algorithms and optimizations, including all-reduce, all-to-all, and all-gather, across scale-up and scale-out networks.
Lead performance tuning, profiling, and analysis of large-scale LLM, diffusion, multimodal, RecSys, and generative AI models on single-node and multi-node systems; drive tradeoffs and design decisions.
Participate in co-design of future scale-up networks, NICs, and scale-out networks, and communicate software performance requirements to hardware teams.
Drive frameworks, tools, and infrastructure for performance estimation, modeling, regression tracking, and reporting.
Work with internal GPU library, compiler, and runtime teams and open-source maintainers, including PyTorch, JAX, vLLM, and SGLang, to align and integrate optimizations.
Provide guidance to customers and partners on network load balancing, workload scheduling, and model sharding strategies.
Present performance analysis and modeling results to stakeholders and senior leadership, with concrete recommendations.
Apply sound engineering principles and establish standards for robust, maintainable solutions.
Provide strategic direction and mentorship to junior team members through code reviews, knowledge sharing, and technical guidance.