Google DeepMind is looking to solve the problem of ensuring the safety, performance, and ethical alignment of their Gemini family of models and associated product applications before and after deployment by developing robust data pipelines, evaluation frameworks, and metric systems.
Requirements
- Experience with large-scale machine learning systems, data processing pipelines and evaluation methodologies.
- Experience with large language models (LLMs) and their evaluation.
- Experience in post-training evaluation research.
Responsibilities
- Work on post-training evaluation and fine-tuning of large-scale models to improve performance and safety.
- Define and champion the technical roadmap for large-scale data and evaluation supporting the Gemini model family and its real-world applications
- Drive the research of novel, high-signal evaluation methods (automated, human-in-the-loop, and adversarial) to measure model capabilities, alignment, safety, and trustworthiness.
- Actively contribute to the broader scientific community by presenting findings on cutting-edge AI evaluation and safety methods.
- Architect and execute the rigorous evaluation and data systems that underpin all major model release and product launch decisions for Gemini.
- Define the data strategy for critical evaluation campaigns.
- Design novel metrics to measure safety and performance at scale.
Other
- Principle level Research Engineer
- key technical leader and individual contributor
- highly cross-functional role requiring a blend of deep ML research, world-class software engineering, and strategic influence.
- mentor a team of engineers and researchers to build high-quality, reproducible systems.
- communicating complex evaluation results directly to leadership stakeholders to guide the responsible deployment of our most advanced AI technology.
- 10+ years of experience in researching engineering, with at least 5 years in a technical leadership role.
