The Testing & Assurance Engineer will design and execute comprehensive verification, validation, and assurance workflows to ensure agentic AI systems are reliable, compliant, and ready for secure deployment in NGA CORE and other National Geospatial-Intelligence Agency (NGA) environments.
Requirements
- 3+ years of hands-on experience in software/system testing, validation, and assurance of AI/ML systems.
- Strong proficiency in Python for building automated test suites, analysis tools, and reporting frameworks.
- Experience designing regression and performance tests for distributed microservice systems.
- Familiarity with AI/ML model evaluation, scoring metrics, and explainability methods.
- Knowledge of software assurance, IV&V, and compliance processes in secure environments.
- Experience with model and agentic AI assurance frameworks, benchmark design, and test harness development.
- Familiarity with ATO/re-accreditation pipelines, automated compliance validation, and RMF/NIST 800-53 security controls.
Responsibilities
- Design and maintain benchmark datasets and test scenarios to validate agentic AI system behavior under real-world conditions.
- Track performance metrics over time to detect accuracy drops, bias emergence, or unintended behavior.
- Develop and execute detailed component-level test plans for individual models, agents, data pipelines, and microservices.
- Verify compliance with interface contracts (APIs, schemas, protocols) and system requirements prior to release.
- Integrate automated tests into CI/CD pipelines (GitLab CI/GitHub Actions/Jenkins) to enable early defect detection.
- Produce validation artifacts, test results, and documentation required for Authority to Operate (ATO) packages and re-accreditation reviews.
- Work closely with AI/ML engineers, agentic systems engineers, and DevSecOps teams to incorporate testability into system design.
Other
- Bachelor’s degree in Computer Science, Software Engineering, or a related field; additional years of experience may be substituted.
- Master’s degree in Software Engineering, Testing & QA, or a related field.
- Operating in NGA CORE or other NGA/Intelligence Community (IC) secure data environments.
- Active TS/SCI clearance (or ability to obtain one).
- U.S Citizenship
