The DLA is looking to establish and implement robust methods for measuring and evaluating AI technologies, ensuring they meet legal, ethical, and performance standards, and advising stakeholders on technical requirements for AI systems.
Requirements
- Utilizes expert techniques and understanding of the complex interrelationships of multiple IT specialties, data science techniques, statistical analysis, and analytical methods to ensure integration of IT programs and services adhere to AI.
- Determines appropriate evaluation metrics to measure model accuracy, fairness, transparency, bias, and robustness.
- Ensures AI testing, evaluation, verification, and validation provide a feedback mechanism for system improvement in building confidence in the use of systems and meeting standards, reliability, and safety requirements.
- Develops illustrations and visualizations of results that can easily be comprehended and simplified for non-technical stakeholder audiences and for communication statistical modeling results as measures of the business's impact.
Responsibilities
- Conducts research and development of metrics, measurements, and evaluation methods for emerging and existing assigned areas of AI.
- Assists in the development of standards; and promotes the adoption of standards, guides, best practices, and PAI policy for measuring and evaluating AI technology projects and/or program segments.
- Ensures AI assigned systems/projects are built in accordance with applicable guidance and meet desired objectives and adhere to the legal, ethical, and performance standards.
- Execute developed comprehensive frameworks for testing the AI systems’ algorithms, models, data, bias, security, and overall performance.
- Incorporates, Office of Management and Budget (OBM) test and evaluation requirements, DOD AI ethical principles into assigned DLA AI test and evaluation framework to generate an RAI test strategy for modeling cases.
- Tests the models to ensure they meet design specifications, business process requirements, data requirements to ensure the AI systems adhere to policy using metrics such as precision, recall, and other Key performance Indicators.
- Advises stakeholders and customers on the technical requirements needed for assigned DLA’s data an AI systems and analytics stack to support data science and ML/AI solutions.
Other
- Telework Eligible
- To qualify for a Data Scientist (AI), your resume and supporting documentation must support: A. Specialized Experience: One year of specialized experience that equipped you with the particular competencies to successfully perform the duties of the position and is directly in or related to this position.
- To qualify at the GS-15 level, applicants must possess one year of specialized experience equivalent to the GS-14 level or equivalent under other pay systems in the Federal service, military, or private sector.
- Applicants must meet eligibility requirements, minimum qualifications, and any other regulatory requirements by the cut-off/closing date of the announcement.
- Creditable specialized experience includes: Utilizes expert techniques and understanding of the complex interrelationships of multiple IT specialties, data science techniques, statistical analysis, and analytical methods to ensure integration of IT programs and services adhere to AI.
