Senior Product Manager –AI/ML Inference Software

Advanced Micro Devices, Inc•Santa Clara, CA

98d•Hybrid

About The Position

This role owns the framework-layer inference product strategy for ROCm, translating customer, ecosystem, and engineering signals into roadmap decisions for production AI inference on AMD Instinct hardware. As inference becomes the defining workload for production AI, you will help shape how AMD’s software ecosystem enables efficient, reliable, and competitive large-scale model deployment. You will work with engineering, strategic AI customers, ecosystem partners, and the open-source inference community to advance inference at scale. You are a technically deep product leader with strong expertise in AI inference infrastructure and open-source software. You can reason across inference engines, serving, orchestration, memory management, and performance while understanding how these systems interact with the GPU software and hardware beneath them. You navigate complex organizations, build alignment without formal authority, and drive important work to completion. You are comfortable operating at the intersection of open-source communities and enterprise-scale customers, whether digging into a GitHub issue thread or presenting roadmap tradeoffs to a VP of Engineering at a hyperscaler.

Requirements

Deep understanding of production AI inference systems, including inference engines such as vLLM, SGLang, and AMD ATOM, and serving and orchestration systems such as llm-d, along with memory management and distributed inference.
Experience driving complex technical initiatives across multiple engineering teams or organizations.
Experience working with sophisticated AI infrastructure customers, open-source communities, or ecosystem partners.
Experience in AI inference infrastructure, GPU computing, distributed systems, cloud infrastructure, high-performance computing, developer platforms, research, or technical product leadership.
Ability to communicate technical tradeoffs clearly to engineering leaders, customer stakeholders, open-source contributors, and executive audiences.
Bachelor’s degree in Computer Science, Electrical Engineering, or a related technical field.

Nice To Haves

Open-source contributions, technical writing, conference talks, or sustained community engagement are strong positive signals.
Product management experience in deeply technical products is valuable but not required. Candidates from engineering, research, or other technical leadership backgrounds are encouraged if they demonstrate strong product instincts and interest in product leadership.
Familiarity with AMD Instinct™ hardware and the ROCm software ecosystem is a plus.
Advanced degree a plus but not required given equivalent experience.

Responsibilities

Own the product strategy and roadmap for ROCm’s inference frameworks software capabilities.
Define how AMD’s framework-layer software stack for inference enables production workloads from single-GPU through rack-scale deployments, with a focus on performance, developer experience, portability, and competitive differentiation.
Identify emerging model architectures, serving technologies, and infrastructure shifts that require changes in the inference frameworks layer and translate them into product priorities.
Balance customer needs, ecosystem direction, technical opportunities, and engineering investment to determine where AMD should differentiate in the software stack.
Partner with engineering teams across inference engines, serving and orchestration, memory systems, GPU libraries, runtimes, kernels, and communications, with particular emphasis on the framework and orchestration layers that enable large-scale inference above the underlying GPU software stack.
Translate customer and ecosystem needs into prioritized engineering requirements and drive execution across organizational boundaries.
Work with engineering to identify inference performance bottlenecks and determine where framework-level software investment can have the greatest impact.
Provide software-informed input to future GPU architecture and platform decisions relevant to inference performance at scale.
Drive release readiness across a fast-moving open-source software ecosystem, with attention to performance, compatibility, regression risk, and adoption.
Serve as an active AMD presence in the open-source inference community.
Build relationships with maintainers, developers, researchers, and contributors whose work influences AI infrastructure and AMD adoption.
Track emerging technologies and ecosystem direction through open-source projects, research, technical communities, and developer feedback.
Communicate AMD’s roadmap and technical direction through community engagement, technical writing, and industry events.
Represent AMD constructively in technical discussions where ecosystem and company priorities may differ.
Work directly with sophisticated AI infrastructure customers to understand deployment requirements, performance constraints, and future needs at the framework and serving layer.
Distinguish individual customer requests from broader technical and market signals, then translate those signals into scalable product decisions.
Balance the needs of strategic customers with those of the broader open-source ecosystem.
Collaborate with cloud providers, model developers, infrastructure companies, and other ecosystem partners on joint integrations and solutions.