Ensuring the quality, reliability, and ethical performance of AI-driven products for Steampunk's Federal contracting clients.
Requirements
- 2+ years of experience in QA, software testing, or validation (AI/ML testing preferred).
- Strong understanding of machine learning concepts and common evaluation metrics.
- Experience with testing frameworks (e.g., PyTest, JUnit) and automation tools.
- Familiarity with Python, SQL, or scripting for test automation.
- Experience working with large language models, computer vision, or recommendation systems.
- Knowledge of data ethics, bias testing, and explainability techniques.
- Hands-on experience with ML frameworks (TensorFlow, PyTorch, Scikit-learn).
Responsibilities
- Develop and execute comprehensive test plans for AI/ML models, including functional, regression, and edge case testing.
- Design test cases to evaluate performance, accuracy, bias, fairness, and robustness of AI outputs.
- Conduct adversarial and stress testing to assess model resilience under unexpected inputs.
- Document and report defects, inconsistencies, and unexpected behaviors.
- Track key quality metrics such as precision, recall, F1 score, false positives/negatives, and usability metrics.
- Validate compliance with data privacy, security, and ethical AI standards.
- Continuously update test frameworks and methodologies to align with advances in AI technology.
Other
- Bachelor’s degree in Computer Science, Data Science, Engineering, or related field (or equivalent practical experience).
- Excellent analytical, problem-solving, and documentation skills.
- Strong attention to detail with a curiosity for breaking things and finding edge cases.
- Clear communication skills to explain findings to both technical and non-technical stakeholders.
- Proactive mindset with the ability to anticipate risks and suggest improvements.
