The Consumer Experience Operations (CXO) team at YouTube is responsible for developing user-centric support solutions that address the evolving needs of YouTube's user base, and is seeking an AI Testing Lead to ensure through rigorous testing programs that conversational and user interfaces provide exceptional user experiences and align with key business objectives.
Requirements
- Experience with testing processes like automated testing, adversarial testing, regression testing.
- Experience working with UX designers, product managers, and engineers.
- Experience with AI based product testing.
- Understanding of UX design principles or conversational AI concepts.
- Ability to interpret data and translate it into insights.
Responsibilities
- Develop a comprehensive test strategy and define the scope of testing, identify types of tests needed, and creating a plan for continuous evaluation of the model in a production environment.
- Analyze data from AI model performance and user interactions, e.g., conversation transcripts, escalation rates, and user flows, to identify pain points and opportunities for design and technical improvements.
- Work with UX and Conversation Designers to ensure design solutions are aligned with business key performance indicators (KPIs), technical constraints, and long-term product strategy.
- Create clear documentation, including test cases, user stories, process flows, and requirements matrices, to ensure a shared understanding across the product, design, and engineering teams.
- Manage a team of testers, overseeing testing requirements and implementing a prioritized feedback loop for tech and design.
Other
- Bachelor’s degree or equivalent practical experience.
- 5 years of experience as a testing lead, business analyst or similar role in the AI domain.
- Excellent communication skills, with the ability to effectively communicate information to both technical and non-technical audiences.
- Ability to work at the intersection of cutting-edge technology and boundless creativity.
- Commitment to equal employment opportunity regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity or Veteran status.
