Google is looking to ensure the safety and integrity of its newest and most advanced AI products by designing insightful evaluation strategies for technologies where standard testing protocols do not yet exist or apply. This involves owning structured pre-launch safety, neutrality, and testing from end-to-end for Google's GenAI products, and inventing the future of testing itself.
Requirements
- Experience using data to provide solutions and recommendations and working with multiple large datasets.
- Experience working with Google's products and services, particularly Generative AI products.
- Understanding of AI systems, machine learning, and their potential risks.
- Ability to think and identify emerging threats and vulnerabilities.
- Experience in trust and safety, product policy, privacy and security, legal, compliance, risk management, intel, content moderation, red teaming, AI testing, adversarial testing, or similar.
- Experience in data analytics or research, business process analysis, or global program management, or leading cross-functional process improvements.
- Translate safety policies into concrete test protocols.
Responsibilities
- Advise the evolution of AI testing by inventing and scaling novel evaluation methodologies and reusable frameworks, collaborating with engineering to enhance testing infrastructure.
- Take ownership of end-to-end evaluations for pioneering GenAI products, translating safety policies into concrete test protocols.
- Design sophisticated prompt strategies and conduct quantitative and qualitative analysis to identify systemic risks and provide insights for launch decisions.
- Partner with Trust and Safety experts to align on standards, developing sophisticated prompt strategies, and leveraging data analysis to provide insights on potential risks.
- Develop and scale testing methodologies, partnering with engineering teams to design the novel infrastructure and tools required to automate and scale.
- Invent and scale novel evaluation methodologies and reusable frameworks.
- Collaborate with engineering to enhance testing infrastructure.
Other
- Bachelor's degree or equivalent practical experience.
- Excellent problem-solving and critical thinking skills with attention to detail in an ever-changing environment
- Excellent communication and presentation skills and the ability to influence cross-functionally at various levels.
- Manage multiple stakeholders through effective communication, and streamline processes.
- Be a subject matter expert, offer expert consultation, drive alignment across Trust and Safety teams, and lead strategic programs to improve the overall testing ecosystem and advance AI safety practices across the organization.
