YouTube's Consumer Experience Operations (CXO) team is looking to solve the business and technical problem of developing user-centric support solutions for evolving user needs across various YouTube products (NFL Sunday Ticket, YouTube TV, YouTube Music and Premium, Shopping, and Fan Funding). The goal is to improve the overall consumer support experience and drive customer lifetime value by identifying and addressing user pain points through rigorous testing programs for conversational and user interfaces, ensuring alignment with business objectives and data-informed design decisions.
Requirements
- Experience with testing processes like automated testing, adversarial testing, regression testing.
- Experience with AI based product testing.
- Understanding of UX design principles or conversational AI concepts.
- Ability to interpret data and translate it into insights.
Responsibilities
- Develop a comprehensive test strategy and define the scope of testing, identify types of tests needed, and creating a plan for continuous evaluation of the model in a production environment.
- Analyze data from AI model performance and user interactions, e.g., conversation transcripts, escalation rates, and user flows, to identify pain points and opportunities for design and technical improvements.
- Create clear documentation, including test cases, user stories, process flows, and requirements matrices, to ensure a shared understanding across the product, design, and engineering teams.
- Manage a team of testers, overseeing testing requirements and implementing a prioritized feedback loop for tech and design.
- Serve as the crucial link between business stakeholders, tech team, and our design team, ensuring through rigorous testing programs that our conversational and user interfaces not only provide exceptional user experiences but also align with key business objectives.
- Translate high-level business goals into requirements and using data to inform and validate design decisions.
Other
- 5 years of experience as a testing lead, business analyst or similar role in the AI domain.
- Experience working with UX designers, product managers, and engineers.
- Excellent communication skills, with the ability to effectively communicate information to both technical and non-technical audiences.
- The US base salary range for this full-time position is $110,000-$157,000 + bonus + equity + benefits.
- Preferred working location from the following: New York, NY, USA; San Bruno, CA, USA
