Pearson is seeking a highly technical GenAI/LLM Systems Test Engineering Manager to lead quality engineering efforts for the XL+ initiative — a flagship program delivering next-generation AI-powered learning solutions such as the Personal Teaching Assistant (PTA), Personal Learning Assistant (PLA), and Open Educational Resource (OER) support. The ideal candidate will design and drive innovative test strategies for AI solutions, ensuring accuracy, reliability, scalability, and trust in our GenAI-powered educational products.
Requirements
- Strong technical expertise in AI/ML product testing, including GenAI workflows, model evaluation, and/or LLM-based solutions.
- Hands-on experience with test automation frameworks, API/UI testing, and CI/CD pipelines.
- Familiarity with LLM testing challenges (e.g., prompt variation, non-deterministic outputs, evaluation metrics, model drift).
- Knowledge of Python or similar languages for building test tools or harnesses.
- Experience in performance, scalability, and security testing of distributed systems.
- Exposure to prompt engineering, bias testing, or AI ethics frameworks is a plus.
Responsibilities
- Define and drive AI Test Strategy: Create comprehensive test strategies specifically for GenAI products and LLM-based workflows, including model accuracy, bias, safety, and reliability validation.
- Develop and maintain AI testing frameworks, including prompt testing and evaluation of model outputs
- Hands-on Technical Leadership: Build and evolve test automation frameworks, pipelines, and evaluation harnesses for AI/ML models, APIs, and integrated systems.
- AI-Specific Quality Validation: Design tests around prompt engineering, hallucination detection, model evaluation metrics, and edge-case scenarios.
- System & Integration Testing: Validate end-to-end workflows, including multi-agent orchestration, MCP models, APIs, and UI interactions.
- Work closely with Performance & Scalability Test tesm to ensure AI-driven systems perform consistently at scale across different use cases and data sets.
- Cross-Functional Collaboration: Partner with Dev, Product, and Data Science to align quality standards with evolving GenAI architectures.
Other
- 6+ years of experience in Quality Engineering, with at least 3 years in a technical leadership or managerial role.
- Mentorship: Grow and mentor a team of QE engineers, building technical depth and expanding the team’s expertise in AI testing.
- Continuous Improvement: Drive defect analysis, implement lessons learned, and adopt emerging tools and methods for testing AI.
- Coordinate testing activities across globally distributed QE teams
- Manage defect tracking, root cause analysis, and continuous improvement for all releases
