Google's Home team is experiencing an increase in investment in AI features, leading to greater demands for evaluations and data. The Scaling team needs to provide essential scaling solutions to make these foundational processes feasible in terms of time and resources, leveraging LLM-based modeling to accelerate and scale solutions beyond manual human judgment/labeling tasks.
Requirements
- 5 years of experience with software development in one or more programming languages.
- 3 years of experience testing, maintaining, or launching software products, and 1 year of experience with software design and architecture.
- Experience using Python libraries and frameworks, especially with ML/AI/Data focus.
- Experience with Natural Language Processing, Machine Learning or Generative AI.
- Experience in data/statistics, leading AI-driven insights and metrics.
- 5 years of experience with data structures/algorithms in a data ML-focused role.
- Experience with scaled batch processing.
Responsibilities
- Lead the work on Generative AI-driven actionable insights, metrics, dashboards, surfacing quality improvement signals.
- Contribute to development of LLM-based automation of human labeling and decision-making tasks via Auto-raters.
- Partner with Home platform teams developing various AI-driven smart home experiences to provide tailored eval solutions.
- Work on extending cutting-edge AI eval frameworks and tooling to help deliver state-of-the-art smart home experiences.
Other
- Mountain View, CA, USA; New York, NY, USA
- Bachelor’s degree or equivalent practical experience.
- Master's degree or PhD in Computer Science or related technical field.
- Experience with evals for Generative AI systems.
- Google is proud to be an equal opportunity workplace and is an affirmative action employer.
