Research Scientist Intern, Foundation Model Optimization (PhD)

Meta•Menlo Park, CA

1d•$7,650 - $12,667

About The Position

Meta's Efficiency and On-Device AI team within the Reality Labs organization is seeking a Research Scientist Intern focused on language and vision model optimization. The primary objective is to develop efficient large language models (LLMs) and vision foundation models. The work includes conducting research intended to result in publications at top-tier computer vision, natural language processing, and machine learning conferences, and developing algorithms and infrastructure to deploy models in resource-constrained environments such as mobile phones and AR/VR/MR/SG devices. The role also includes investigating joint optimization of models and assistant systems, including planning, memory, tool use, and coordination across device and cloud; exploring personalization, model compression and efficiency, and novel architectures such as omni-modal capabilities; and collaborating with research and engineering teams.

Requirements

Currently be in the process of obtaining a PhD in Artificial Intelligence or a related field.
Obtain work authorization in the country of employment by the time of hire and maintain ongoing work authorization during employment.
Hands-on experience with deep learning algorithms and techniques, such as transformers.
Experience with deep learning software libraries such as PyTorch or JAX.
Experience with scripting languages such as Python and shell scripts.

Nice To Haves

Intent to return to a degree program after completing the internship.
Experience with large-scale foundation model training and on-device foundation model optimization.
Experience in one or more of the following: LLM agents and orchestration, personalization, model compression and efficient inference, efficient model architectures, multimodal modeling, or data-centric approaches for model tuning and evaluation.
A proven publication record at CV/NLP/ML conferences or journals such as NeurIPS, ICLR, ICML, CVPR, ICCV, ECCV, AAAI, ACL, or EMNLP.

Responsibilities

Perform LLM and vision foundation model research intended to lead to publications in top-tier CV/NLP/ML conferences.
Develop model optimization algorithms and infrastructure to enable deployment in resource-constrained settings, including mobile phones and AR/VR/MR/SG devices.
Research joint optimization of models and surrounding assistant systems, including planning, memory, tool use, and coordination across device and cloud.
Explore personalization, model compression and efficiency, and novel model architectures, including omni-modal capabilities.
Collaborate with cross-disciplinary research and engineering teams.