The company is looking to enhance the development experience for teams working on Gemini features by ensuring their quality assessments.
Requirements
- Experience programming in C++.
- Experience developing or maintaining backend systems, data pipelines, or large-scale software infrastructure, including experience with SQL.
- Experience with evaluation methodologies, statistical analysis, or quality assurance, especially in the context of Machine Learning or AI systems.
- Experience designing, building, and maintaining backend infrastructure, particularly for data processing or evaluation systems.
- Experience in logging systems, log analysis, and data processing tools to diagnose issues.
Responsibilities
- Design, build, and continuously improve backend infrastructure and pipelines focused on evaluation reliability, replayability, and quality monitoring, leveraging system/log analysis.
- Develop and manage metrics, dashboards, and monitoring systems to track and ensure the trustworthiness and consistency of evaluation results.
- Collaborate closely with feature engineers, quality leads, and other stakeholders across the product area to define requirements and prioritize infrastructure development.
- Collaborate with peer engineers to define technical strategy, unify evaluation stacks, and establish best practices for evaluations.
- Own evaluation infrastructure components, driving development and improvements throughout the project lifecycle.
Other
- Bachelor’s degree in Computer Science, a related technical field, or equivalent practical experience
- 2 years of professional software development experience, OR 1 year of experience with an advanced degree (e.g., Master's, PhD) in a relevant field.
- 2 years of experience applying fundamental data structures and algorithms to solve technical problems.
- Master’s degree or PhD in Computer Science or a related technical field.
