Software Engineer III, AI/ML

Google•New York, NY

9h•$147,000 - $210,000

About The Position

Google software engineers develop technologies and products that serve billions of users and operate at massive scale. In this role, you will work on a project important to Google, with opportunities to move between teams and projects as business needs evolve. The position calls for versatility, leadership, and willingness to tackle new problems across the full stack. The role is associated with Gemini Encoder-heavy, which provides production-ready Gemini-based encoder-heavy models, architectures, pipelines, and optimization recipes. These models are used across Google product areas, including Search, Ads, and YouTube, and the codebase is the main infrastructure powering Gemini Embeddings. The position focuses on developing and optimizing encoder-heavy models and frameworks, including model training, efficiency, and support for evolving client needs. Individual pay is determined by factors such as job-related skills, experience, and relevant education or training. The listed U.S. compensation is $147,000–$210,000, plus a 15% bonus target, equity, and benefits.

Requirements

Bachelor’s degree or equivalent practical experience.
At least 2 years of software development experience in one or more programming languages, or 1 year of experience with an advanced degree.
Experience developing, modifying, or training core internal Transformer components for foundation language models, such as custom or sparse attention mechanisms, sequence compression, Mixture-of-Experts, or encoder/decoder backbones.
Experience pre-training or fine-tuning language and multimodal embedding models using techniques such as Supervised Fine-Tuning, contrastive learning, or parameter-efficient methods.
Experience with neural network compression and accelerator-level GPU or TPU inference optimization, such as quantization, structured or expert pruning, or memory-latency trade-off tuning.

Nice To Haves

Master’s degree or PhD in Computer Science or a related technical field.
At least 2 years of experience with data structures and algorithms.
Proficiency in Python and experience with deep learning frameworks such as JAX.
Strong understanding of Transformer-based architectures, attention mechanisms, and the distinction between encoder and decoder models.

Responsibilities

Write product or system development code.
Collaborate with peers and stakeholders through design and code reviews to ensure best practices in areas such as style guidelines, code submission, accuracy, testability, and efficiency.
Contribute to the development and optimization of an encoder-heavy framework based on Gemini.
Engineer training recipes to maximize model efficiency, targeting latency, memory footprint, and compute.
Communicate with stakeholders to triage technical issues and develop features that address evolving client needs.