Senior Machine Learning Engineer - Foundation Model

XPENG • Santa Clara, CA

1d • $174,720 - $295,680

About The Position

XPENG is a leading smart technology company focused on integrating advanced AI and autonomous driving technologies into its vehicles. This role involves driving the modeling and algorithmic development of XPENG’s next-generation Vision-Language-Action (VLA) Foundation Model, which powers end-to-end autonomous driving systems. The engineer will collaborate with researchers and engineers to design, train, and deploy large-scale multi-modal models unifying vision, language, and control, directly shaping the intelligence for future L3/L4 autonomous driving products.

Requirements

• Master’s degree or higher in Computer Science, Electrical/Computer Engineering, or related field, with 3+ years of experience in deep learning research or productization.
• Strong proficiency in PyTorch and modern transformer-based model design.
• Experience in large-scale pretraining or multi-modal modeling (vision, language, or planning).
• Deep understanding of representation learning, temporal modeling, and self-supervised or reinforcement learning techniques.
• Familiarity with distributed training (DDP, FSDP) and large-batch optimization.

Nice To Haves

• PhD in CS/CE/EE or related field, with 1+ years of relevant industry experience.
• Publication record in top-tier AI conferences (CVPR, ICCV, NeurIPS, ICLR, ICML, ECCV).
• Prior experience building foundation or end-to-end driving models, or LLM/VLM architectures (e.g., ViT, Flamingo, BEVFormer, RT-2, or GRPO-style policies).
• Familiarity with RLHF/DPO/GRPO, trajectory prediction, or policy learning for control tasks.
• Proven ability to collaborate cross-functionally with infra, perception, and planning teams to deliver production-ready models.

Responsibilities

• Design and implement large-scale multi-modal architectures (e.g., vision–language–action transformers) for end-to-end autonomous driving.
• Develop pretraining and fine-tuning strategies leveraging massive labeled and unlabeled fleet data (images, video, LiDAR, CAN bus, maps, human driving behaviors, etc.).
• Research and integrate cross-modal alignment (e.g., visual grounding, temporal reasoning, policy distillation, imitation and reinforcement learning) to improve model interpretability and action quality.
• Collaborate with infrastructure engineers to scale training across thousands of GPUs using distributed training frameworks (FSDP, DDP, etc.).
• Conduct systematic ablation, evaluation, and visualization of model behavior across perception, reasoning, and planning tasks.
• Contribute to model deployment optimization, including quantization, export, and latency–accuracy trade-offs for onboard execution.

Benefits

• A collaborative, research-driven environment with access to massive real-world data and industry-scale compute.
• An opportunity to work with top-tier researchers and engineers advancing the frontier of foundation models for autonomous driving.
• Direct impact on the next generation of intelligent mobility systems.
• Opportunity to make significant impact on the transportation revolution by the means of advancing autonomous driving.
• Competitive compensation package.
• Snacks, lunches, dinners, and fun activities.
• Bonus
• Equity
• Benefits

What This Job Offers

• Job Type: Full-time
• Career Level: Senior