Research Engineer, Human Likeness Modeling

DeepMind•New York, NY

About The Position

We are seeking a highly motivated Research Engineer (L5) with a strong background in multi-modal modelling and a focus on speech/audio to join the Human Likeness Modeling (HLM) effort within Google DeepMind's Frontier AI unit. This role is pivotal in developing foundational multimodal AI capabilities to understand, generate, and protect human likeness. As a key contributor, you will design and implement cutting-edge models and frameworks, pushing the boundaries of AI to enable foundational capabilities with a focus on human-centric understanding and generation. This is a unique opportunity to contribute to impactful research and advance Google DeepMind's mission towards Artificial General Intelligence (AGI). About us Artificial Intelligence could be one of humanity’s most useful inventions. At Google DeepMind, we’re a team of scientists, engineers, machine learning experts and more, working together to advance the state of the art in artificial intelligence and ultimately achieve Artificial General Intelligence. We use our technologies for widespread public benefit and scientific discovery, and collaborate with others on critical challenges, ensuring safety and ethics are the highest priority. The effort is a part of Google DeepMind's Frontier AI unit. The team aims to build holistic representation encompassing a full spectrum of human understanding. We develop systems to provide perception skills critical for person-centric applications, which is crucial for enabling AI to interact naturally & seamlessly, depict humans accurately & responsibly in generative AI, and build trustworthy & resilient systems that can detect and prevent misuse like deepfakes and impersonation. The role You will drive outcomes for critical technical components aimed at advancing our capabilities in multimodal human understanding. You will play a critical role in developing and deploying models that can provide accurate human understanding across multiple modalities (e.g., visual appearance, voice, dynamics, etc), while also building robust defenses against sophisticated AI-driven manipulation and impersonation. This role involves tackling complex, ambiguous problems with no obvious "best" solution, requiring independent judgment and a proactive approach to exploring multiple technical avenues. You will be instrumental in shaping the technical direction for core components of the effort. Your contribution will lead to key breakthrough and impactful landings within GDM and across Google products, ensuring our technologies are both groundbreaking and responsibly deployed.

Requirements

PhD degree in Computer Science, Machine Learning, or a related technical field with 3+ years of relevant experience.
Experience in developing machine learning models, such as audio & speech-visual models.
Experience in working with and tuning large-scale vision language models.
Strong programming skills in Python and experience with at least one major deep learning framework (e.g., JAX)
Experience conducting independent research and development, including experimental design, implementation, and analysis.

Nice To Haves

Experience with Generative AI techniques and architectures.
Familiarity with Reinforcement Learning or alignment methods.
A track record of publications in top-tier AI/ML conferences (e.g., NeurIPS, ICML, ICLR, CVPR, ICCV, ECCV).
Experience with multimodal learning, integrating information from different data types (e.g., vision, audio, text).
Understanding of privacy-preserving machine learning or responsible AI practices.

Responsibilities

Advance multimodal human representations & understanding : Research and implement novel models and other multimodal techniques for a more holistic understanding of humans across visual, audio, and textual data.
Conduct applied research: Conduct experimental research cycles from hypothesis to deployment.
Drive technical projects: Take ownership of substantial technical projects within the effort, from ideation and design to implementation and evaluation, often involving cross-functional collaboration.
Contribute to Infrastructure: Inform and contribute to the development of scalable and efficient research infrastructure for multimodal human understanding models and datasets.
Design and execute strategies for tuning and adapting VLMs and other foundation models for specific tasks