Zoox is developing the first ground-up, fully autonomous vehicle fleet and the supporting ecosystem required to bring this technology to market. This role focuses on the Perception system, the "Retina of Zoox", which is responsible for understanding the world around the autonomous vehicle.
Requirements
- Advanced understanding of Python or C++ (C++ preferred)
- Experience with production ML pipelines: dataset creation, labeling, training, metrics
- Experience training/finetuning MLLMs or at least MLLms (SFT/RL)
- Experience with Vision-Language Models
- Experience with model deployment with TensorRT
- Experience with Neural Network design and implementation
- Experience working with LiDAR, Camera and Radar data
Responsibilities
- develop advanced multimodal large language models that enhance scenario understanding and driving
- develop and fine-tune models with driving data, ensuring models can efficiently identify hazards, interpret driving restrictions, drive and answer questions about the scenario
- collect and generate datasets for specialized vehicle classification and semantic enrichment
- design and frame machine learning problems for real-world autonomous driving scenarios and train and evaluate state-of-the-art machine learning models with a focus on computer vision
- collaborate with engineers to deploy models for real-time inference on our vehicles, and contribute to improving our vehicle's ability to recognize and respond to emergency vehicles, school buses, construction vehicles, and other specialized road actors
- develop advanced ML models that perceive our vehicle's surroundings to identify hazards and driving restrictions
- develop multimodal foundation models that serve as the common backbone for on-vehicle perception, enhancing the system's ability to detect long-tail events and generalize to new geofences
Other
- Currently working towards a B.S., M.S., Ph.D., or advanced degree in a relevant engineering program
- Must be returning to school to continue your education upon completing this internship
- Good academic standing
- Able to commit to a 12-week internship beginning in May or June of 2026.
- At least one previous industry internship, co-op, or project completed in a relevant area
