Backend Software Engineer, Machine Learning Platform, AI Infrastructure

Tesla•Palo Alto, CA

About The Position

As a Software Engineer within the Autopilot AI Infrastructure team, you will work on reinforcing, optimizing, and scaling our infrastructure components supporting AI research activities for Autopilot and the Optimus. At the core of our autonomy capabilities are neural networks that the research team is designing to train on very large amounts of data, across large-scale GPU clusters. Robustly training these models at scale and in the shortest amount of time is critical to our mission. We are building out the Machine Learning Platform that our engineers and leadership use to schedule, manage and monitor machine learning experiments, data pipelines and artifacts. With the ever-increasing size of our datasets and compute clusters, we are looking for an experienced backend engineer to help drive scalability improvements and new capabilities in the platform.

Requirements

Expertise in designing scalable and durable distributed systems
Strong knowledge of Python/Go and Linux
Experience working with diverse backend infrastructure components (SQL / NoSQL databases, caching, message brokers, event streams, monitoring etc.)
Hands-on experience with containerization and orchestration technologies (Docker, Kubernetes) and setting up CI/CD flows
Knowledge of front-end development in React / strong product sense
Knowledge of machine learning, computer vision, or neural networks
Experience working with HPC clusters

Responsibilities

Develop and deploy solutions to scale our infrastructure effectively in response to rapidly growing demands
Drive implementation of best practices and monitoring systems to proactively detect and address issues in our production environment
Work across the stack on tools and infrastructure empowering the machine learning team to be effective. This ranges from developing/running model training and evaluation code to back-end infrastructure to occasional front-end work
Coordinate required resources with the team managing the cluster hardware to maintain high availability
Work closely with the research team to understand requirements and priorities

Benefits

Along with competitive pay, as a full-time Tesla employee, you are eligible for the following benefits at day 1 of hire:
Medical plans > plan options with $0 payroll deduction
Family-building, fertility, adoption and surrogacy benefits
Dental (including orthodontic coverage) and vision plans, both have options with a $0 paycheck contribution
Company Paid (Health Savings Accounts) HSA Contribution when enrolled in the High-Deductible medical plan with HSA
Healthcare and Dependent Care Flexible Spending Accounts (FSA)
401(k) with employer match, Employee Stock Purchase Plans, and other financial benefits
Company paid Basic Life, AD&D
Short-term and long-term disability insurance (90 day waiting period)
Employee Assistance Program
Sick and Vacation time (Flex time for salary positions, Accrued hours for Hourly positions), and Paid Holidays
Back-up childcare and parenting support resources
Voluntary benefits to include: critical illness, hospital indemnity, accident insurance, theft & legal services, and pet insurance
Weight Loss and Tobacco Cessation Programs
Tesla Babies program
Commuter benefits
Employee discounts and perks program