Hardware/Software Co-Design Engineer, Inference

Google•Sunnyvale, CA

3d•$192,000 - $278,000

About The Position

In this role, you will help shape AI/ML hardware acceleration and drive TPU technology for demanding AI/ML applications. You will contribute to custom silicon solutions and focus on TPU architecture and its integration within AI/ML-driven systems. You will serve as a technical link between model architecture innovation and next-generation hardware design, working across AI research and engineering to shape the architectural roadmap for machine learning serving and training. The role involves integrating ML research, including training and serving massive foundation models, with advanced silicon architectures to deliver high-performance, power-efficient accelerators. The AI and Infrastructure team delivers AI and infrastructure capabilities at scale, efficiency, reliability, and velocity for Google customers, Googlers, and Google users. Its work spans software and hardware, including TPUs, Vertex AI for Google Cloud, Google Global Networking, data center operations, and systems research.

Requirements

Bachelor's degree in Electrical Engineering, Computer Engineering, Computer Science, or a related field, or equivalent practical experience.
10 years of experience in computer architecture, chip architecture, or hardware-software co-design.
Experience developing systems for performance modeling, simulation, or system analysis.

Nice To Haves

Master's degree or PhD in Electrical Engineering, Computer Engineering, or Computer Science, with an emphasis on computer architecture.
Experience architecting hardware solutions or performance optimizations for ML training and inference.
Experience with deep learning frameworks such as TensorFlow or PyTorch.
Deep understanding of ML trends, business drivers, and the software ecosystem.
Ability to engage and collaborate with hardware designers, software architects, and ML researchers.

Responsibilities

Drive the definition and optimization of the hardware/software stack to enable performant training and serving of large ML models.
Collaborate with research and modeling teams on model architectures, focusing on scaling, quality, and their direct impact on hardware performance.
Lead the development of configurable architectural simulators and cycle-accurate performance models to quantify microarchitectural optimizations and evaluate architectural decisions.
Conduct system-level performance analysis across highly distributed ML systems, developing methodologies to balance compute, memory bandwidth, and inter-chip network requirements.
Engage with hardware design, compiler development, and ML research partners to transition architectural innovations from concept to production.