Waymo is seeking to solve the challenge of robustly evaluating large AI models in autonomous driving systems, which is critical for ensuring safety and performance in real-world conditions.
Requirements
- 5+ years of relevant industry experience in a heavily quantitative software engineering area
- Experience navigating complex technical and product landscapes, defining technical strategy, and creating roadmaps.
- Proficiency in programming in Python or C++
- Experience with software design principles, coding best practices, testing methodologies, and version control software.
- Experience building software pipelines for data processing, system evaluation, or metric computation, in the context of large-scale systems.
- Knowledge of AI fundamentals, such as transformer architectures, distillation techniques, etc.
- Experience evaluating the quality of ML models
Responsibilities
- Develop novel metrics and sampling techniques to measure the driving trajectories generated by ML models.
- Employ creative simulation strategies to measure the driving performance of generative AI models. Identify potential edge cases, and provide reliable performance insights that inform model development and deployment.
- Build data pipelines for signal discovery, data labeling, feature extraction and metric computation based on large-scale simulations.
- Conduct data analysis to diagnose regressions in ML models.
- Collaborate with world-class engineering and research teams that develop large-scale ML models.
Other
- Demonstrated experience taking quantitative findings through to productionized tools.
- Experience with simulation systems, robotics, or autonomous vehicles.
- Familiarity with one of the modern deep learning frameworks (e.g. JAX, Tensorflow)
