2027 Summer Intern, PhD, Research, VLM Harnesses for AV Planning
About The Position
Waymo’s AI Foundations team develops machine learning solutions for open problems in autonomous driving, with the goal of safely operating Waymo vehicles in many cities and driving conditions. The team’s research areas include reinforcement learning, learning from demonstration, generative modeling, Bayesian inference, hierarchical learning, and robust evaluation, and it also collaborates with other research teams in Alphabet. As a research intern, you will work on VLM harnesses for autonomous vehicle planning. You will develop ways to present driving scenarios to state-of-the-art vision-language models, translate their outputs into feasible and safe trajectories, and evaluate the systems in closed-loop autonomous driving simulation. You will communicate research motivations and findings to technical and non-technical audiences, and may share findings with the wider academic community through technical reports or publications. The internship is hybrid onsite. Applications are accepted on a rolling basis until the role is filled. The expected hourly rate is $85, and interns may participate in company benefits programs subject to eligibility requirements.
Requirements
- Currently enrolled in a PhD program in computer science, statistics, applied mathematics, physics, or a related technical field of study.
- Familiarity with major machine learning frameworks such as JAX and PyTorch.
- Experience using coding agents for auto-research.
- At least 3 years of experience with deep learning, computer vision, and generative models such as VLMs or LLMs.
Nice To Haves
- Publications at top-tier conferences such as CVPR, ICCV, ECCV, ICLR, ICML, NeurIPS, IROS, or CoRL.
- Substantial involvement in and contributions to high-impact industry AI projects.
- Experience with vision-language models (VLMs), multimodal prompting or reasoning, or autonomous driving simulation and planning.
Responsibilities
- Design and implement scenario preprocessors, including visual rendering, scene representations, and multimodal prompting, to present driving scenarios effectively to state-of-the-art VLMs.
- Develop postprocessors that translate VLM outputs into feasible, safe trajectories for closed-loop vehicle control.
- Evaluate VLM driving harnesses in closed-loop autonomous driving simulation and iterate on representations based on driving metrics.
- Communicate research motivations and findings to technical and non-technical audiences.
- Conduct cutting-edge research and potentially communicate findings to the wider academic community through technical reports and/or publications.
Benefits
- Participation in the Company’s benefits programs, subject to eligibility requirements.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Intern
Education Level
Ph.D. or professional degree
