Microsoft Turing Team is looking to improve the productivity of hundreds of millions of users around the world by developing the core systems that power Microsoft 365 Copilot Chat, delivering intelligence, quality, and transformative new features. The Experimentation & Evaluation area aims to define and measure the success of Copilot by working on metrics, scorecards, and evaluation pipelines.
Requirements
- 4+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C-Sharp, Java, JavaScript, or Python
- 4+ years of experience building AI-powered products, data processing frameworks, or evaluation systems
- 4+ years of experience in evaluation systems
Responsibilities
- build pipelines that power Copilot’s measurement and experimentation
- design reliable systems for offline, shadow, and online scorecards
- enabling fast and trustworthy evaluation of AI models and features
- turn complex data into actionable insights that accelerate Copilot’s innovation and impact
- Creates, implements, optimizes, debugs, refactors, and reuses code to establish and improve performance and maintainability, effectiveness, and return on investment (ROI)
- Acts as a Designated Responsible Individual (DRI) and guides other engineers by developing and following the playbook, working on call to monitor system/product/service for degradation, downtime, or interruptions, alerting stakeholders about status and initiates actions to restore system/product/service for simple and complex problems when appropriate
- Proactively seeks new knowledge and adapts to new trends, technical solutions, and patterns that will improve the availability, reliability, efficiency, observability, and performance of products while also driving consistency in monitoring and operations at scale
Other
- Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include but are not limited to the following specialized security screenings:* *Microsoft Cloud Background Check:
- This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter.
- Collaborates with appropriate stakeholders to determine user requirements for a scenario.
- Drives identification of dependencies and the development of design documents for a product, application, service, or platform.
- Leverages subject-matter expertise of product features and partners with appropriate stakeholders (e.g., project managers) to drive a workgroup's project plans, release plans, and work items.
