Cognizant is looking to expand its team in Seattle, Washington to help clients modernize technology, reinvent processes, and transform experiences. The specific role focuses on ensuring the accuracy and reliability of Large Language Models (LLMs) through validation, testing, and defect analysis.
Requirements
- 3+ years of professional work experience in Blackbox Quality Assurance, with experience in software testing, AI technologies or LLM evaluation
- Knowledge of data analysis & strong understanding of software testing methodologies
- Strong understanding of software testing methodologies
- Proficiency in test planning, test design, test authoring, and execution
- Experience in test planning, test design, test authoring, and execution
- Analyzing test results, identify defects, patterns of issues, troubleshoot, file detailed defect reports
- Experience with Artificial Intelligence, Machine Learning, ie: Chat GBT
Responsibilities
- Working with large language model validation, ensuring the LLM performs at an acceptable level of accuracy and reliability across different models
- Fix implementation: Identify and address issues in synthetic data and manual tests by comparing various models, fixing schemas
- Create, curate diverse set of test cases and scenarios to challenge and refine LLM’s
- Monitor Pipelines, triage failures, triage, and escalate critical pipeline failures
- End to end, functional, regression testing on client software & hardware
- Manual testing with AI, Personal Assistants such as Alexa, Siri, Google
- Perform test planning, test case development, and manual test execution
Other
- Do you thrive in a busy environment and able to multi-task successfully?
- Meticulous with attention to detail
- Possess excellent written and verbal communications skills in English
- Able to work in a fast paced dynamic environment
- Curious and proactive. Loves to learn and explore
