Microsoft's Customer Experience and Success (CE&S) organization is looking to accelerate AI transformation for its customers and the world by establishing and leading an evaluation, testing, and experimentation function for AI solutions. The goal is to ensure AI programs deliver reliable, scalable, and responsible value by implementing a unified framework for comprehensive evaluation, rigorous testing, and safe experimentation.
Requirements
- Proven experience in AI/ML systems architecture and technical leadership.
- Strong knowledge of model evaluation frameworks (e.g., MLflow, TensorBoard) and automated QA tools.
- Expertise in Azure MLOps practices, CI/CD pipelines, and cloud-based AI platforms.
- Familiarity with responsible AI principles, compliance frameworks, and risk mitigation strategies.
- Hands-on experience with large-scale AI deployments and performance optimization.
- Background in data governance and ethical AI implementation.
Responsibilities
- Collaborate with various technical team to validate that technical architecture standards for AI/ML systems across multiple projects to take into account the quality and resilience.
- Establish scalable quality assurance patterns for model deployment and integration
- Develop and maintain frameworks for model performance, fairness, and robustness evaluation.
- Implement quality gates and continuous testing pipelines for AI components.
- Drive development velocity through streamlined processes and MLOps best practices.
- Monitor and optimize workflows to balance speed with compliance and risk mitigation.
- Translate responsible AI principles into actionable technical controls.
Other
- Embody our culture and values.
- 2+ years of experience managing cross-functional and/or cross-team projects.
- Excellent communication and stakeholder management
- Microsoft will accept applications for the role until November 8, 2025.
