Microsoft's Customer Experience & Success (CE&S) organization needs to establish and lead the evaluation, testing, and experimentation function for AI solutions to ensure they deliver reliable, scalable, and responsible value, accelerating AI transformation for customers and the world.
Requirements
- Proven experience in AI/ML systems architecture and technical leadership.
- Strong knowledge of model evaluation frameworks (e.g., MLflow, TensorBoard) and automated QA tools.
- Expertise in Azure MLOps practices, CI/CD pipelines, and cloud-based AI platforms.
- Familiarity with responsible AI principles, compliance frameworks, and risk mitigation strategies.
- Hands-on experience with large-scale AI deployments and performance optimization.
- Background in data governance and ethical AI implementation.
Responsibilities
- Establish scalable quality assurance patterns for model deployment and integration
- Develop and maintain frameworks for model performance, fairness, and robustness evaluation.
- Implement quality gates and continuous testing pipelines for AI components.
- Drive development velocity through streamlined processes and MLOps best practices.
- Monitor and optimize workflows to balance speed with compliance and risk mitigation.
- Translate responsible AI principles into actionable technical controls.
- Ensure adherence to security, privacy, and ethical standards throughout the lifecycle.
Other
- 3 days / week in-office
- Travel 0-25%
- Embody our culture and values.
- Excellent communication and stakeholder management
