Research Scientist Intern, 3D-grounded Generalist Model for Robotics (PhD)

Meta•Redmond, WA

3d•$7,650 - $12,667

About The Position

Meta Reality Labs Research’s Surreal Vision group is seeking a PhD-level Research Scientist Intern to conduct fundamental research on 3D-grounded generalist models for robotics. The work aims to enable accurate and efficient zero-shot robotic manipulation in everyday environments. The intern will tackle research challenges, develop model architectures and training recipes, and deliver general robotics policies. Projects may involve improving the 3D and spatial understanding of large-scale foundation models through data and training recipes; developing interactive, real-time generalist robotics policies based on large-scale 3D foundation models; and validating policies in simulation and on real robots, potentially culminating in a live demonstration on physical hardware. Internships last 12 to 24 weeks, with various start dates throughout the year; some projects may require at least 24 consecutive weeks.

Requirements

Currently pursuing or in the process of obtaining a PhD in computer vision, robotics, machine learning, computer graphics, or a related field.
Must obtain work authorization in the country of employment by the time of hire and maintain work authorization during employment.
Demonstrated knowledge of and hands-on experience with robotics policy training.
Demonstrated knowledge of and hands-on experience with large-scale 3D data pipelines and curation for robotics or foundation model training.
Hands-on experience implementing large foundation models and generative architectures, such as LLMs, VLMs, video diffusion models, world (action) models, VLAs, and reinforcement learning.
Experience with Python and deep learning frameworks such as PyTorch.

Nice To Haves

Ability to commit to 24 consecutive weeks of full-time internship work.
Research achievements demonstrated by grants, fellowships, patents, or first-authored publications, ideally with publicly available code, at premier conferences or workshops such as CoRL, RSS, ICRA, IROS, CVPR, ECCV, ICCV, NeurIPS, ICLR, ICML, SIGGRAPH, or SIGGRAPH Asia.
Strong research background in robotics policy training, large foundation models (VLM/LLM), world (action) models, video diffusion models, or neural 4D reconstruction and rendering.
Strong programming experience using Python and PyTorch.
Software engineering experience through internships, work experience, coding competitions, or open-source contributions such as GitHub repositories.
Experience collaborating and communicating cross-functionally in a team environment.
Intent to return to a degree program upon completion of the internship.

Responsibilities

Plan and execute research advancing generalist robotics policies and data pipelines and curation for large-scale 3D foundation models.
Collaborate with researchers and engineers across Meta robotics teams to develop experiments, prototypes, and concepts for robotics policy training and evaluation.
Design, set up, and run experiments and prototype systems to validate and demonstrate generalist robotics policies based on large-scale 3D foundation models in simulation and on real robots.