MCG is looking to enhance the US healthcare system by developing evidence-based content and innovating products, and the Manager, Data Science role is intended to lead the team responsible for effectively testing agents to achieve this goal.
Requirements
- Expert level Python proficiency
- Ability to write reliable, extensible python packages
- Demonstrated experience translating high level requirements into useful tools
- Excellent knowledge of frequentist and Bayesian statistical techniques
- Experience with basic principles of deep learning and/or natural language processing
- Experience working specifically on automated model evaluation
Responsibilities
- Direct annotation by managing annotation budgets, sourcing data, and providing statistical analysis of annotations
- Maintain and expand metrics used to evaluate agent performance and fairness
- Create evaluation packages for Data Scientist and Machine Learning Engineers to use when running experiments
- Ensure accurate, regular reporting on evaluation metrics to stakeholders
- Manage periodic re-evaluation of production agents using user data and annotation
Other
- Travel expected 3-4 times per year for company-sponsored events
- Remote work with occasional travel (1–3 trips per year)
- Good written communication skills
- Healthcare experience
- Bachelor's degree or higher (not explicitly mentioned but implied)
