Filtr uses AI to scan 1000+ jobs and finds postings that perfectly matches your resume

Waymo is looking to improve the safety and reliability of its autonomous driving technology by rigorously assessing the performance of its Perception systems, a critical component for safe and effective autonomous operation. The goal is to ensure the quality and reliability of Waymo's autonomous vehicles, especially in new and diverse urban environments, and to integrate evaluation findings into the ML development loop for faster system improvement.

Requirements

• Deep expertise in the evaluation of Machine Learning systems at scale, encompassing metrics design, rigorous experimentation, large-scale data analysis, and statistical methods
• Hands-on experience with ML infrastructure, modern AI concepts (including foundation models or similar), and their practical application to solve real-world problems
• Proficiency in C++ and Python in a production environment
• Familiarity with autonomous vehicle systems, particularly Perception, including typical data pipelines and evaluation processes
• Hands-on experience with data analysis, visualization, and triage tools & workflows, with a knack for identifying improvement areas
• Experience in designing and implementing efficient data processing workflows for large datasets
• Familiarity with or experience in developing solutions using Generative AI or building AI agents

Responsibilities

• Lead Advanced Evaluation Framework Design: You will be responsible for the architectural design and development of innovative evaluation frameworks to thoroughly assess perception system performance, particularly in new and diverse urban environments. This includes defining technical specifications and ensuring robustness.
• Innovate with AI in Evaluation Methodologies: You will develop and implement cutting-edge evaluation methodologies and scalable pipelines. This involves leveraging advancements in large-scale AI models (including language and multimodal models) to enhance analysis, generate deeper insights, and pioneer new ways to assess performance.
• Utilize Simulation and Reference Data Systems: You will drive the use of large-scale inference and simulation systems to establish performance benchmarks, validate perception capabilities, and proactively identify potential perception challenges in a variety of scenarios.
• Drive End-to-End Platform Delivery: You will exercise complete end-to-end ownership in delivering a robust and adaptable discovery evaluation platform. This platform must be capable of supporting rapid iteration and meeting the demanding timelines associated with Waymo's city expansions, enabling proactive problem-solving.
• Integrate Evaluation with ML Development Cycles: You will collaborate closely with Machine Learning and Data Infrastructure teams. Your responsibility is to ensure that evaluation findings are seamlessly integrated into the data-driven model development loop, creating a flywheel to accelerate training, hillclimbing, and overall system improvement.
• Resolve Complex Technical Challenges: You will proactively identify, troubleshoot, and resolve complex technical hurdles within the perception evaluation domain. This involves deep technical investigation and ensuring the delivery of high-quality, actionable feedback for system enhancements, often under tight deadlines.

Other

• Demonstrated technical leadership in defining long-term strategy, developing roadmaps, and guiding teams to execute projects from ambiguous requirements to high-impact deliverables
• Demonstrated ability to work effectively across diverse teams and communicate technical concepts clearly
• Excellent problem-solving skills, clear technical communication, and a strong sense of end-to-end ownership
• Hybrid role
• Report to a Staff Technical Lead Manager