The University of Florida's College of Medicine is seeking a Data Scientist II to lead the analysis and reporting of medical education assessments, with a focus on Clinical Skills Examinations (CSEs), and to apply advanced psychometric methodologies to enhance the validity of high-stakes evaluations.
Requirements
- Proficiency in statistical analysis software (e.g., R, Python, SAS, SPSS, Stata)
- Strong skills in data cleaning, management, and quality assurance
- Experience with psychometric procedures and methods, including classical test theory, item response theory, and adaptive testing
- Experience conducting rigorous analyses in medical education or related social/health sciences
- Strong written and verbal communication for research dissemination
- Experience with technical writing (grants and manuscripts)
- Experience leading standard setting, building weighted grading/score models, and producing longitudinal competency profiles in clinical skills and clerkship contexts
Responsibilities
- Create psychometric analyses (CTT, IRT, factor, and generalizability/reliability studies) to evaluate and improve high-stakes assessments.
- Conduct inter-rater reliability analyses to improve validity and provide consistency by reducing score drift.
- Collaborate with the Medical Director of ALAC to design and facilitate standard-setting panels, justify defensible passing scores, and revise grading schemas and scoring models.
- Lead analysis, interpretation, and reporting of performance for medical education assessments, including performance data for the CSE.
- Develop and maintain learning analytics that inform progress, continuous quality improvement and summative reporting to program leadership.
- Design, implement, and analyze educational research projects in collaboration with the Assistant Dean of Medical Education Research, Scholarship, and Faculty Development.
- Prepare and manage regulatory documents (IRB applications, data use agreements) to ensure compliance with research standards.
Other
- A Bachelor’s Degree in data science, statistics, bioinformatics, analytics, or similar field and three years of experience
- Master’s Degree in data science, statistics, bioinformatics, analytics, or similar field and one year of experience
- Doctoral Degree in data science, statistics, bioinformatics, analytics, or similar field
- Demonstrated ability to manage multiple complex projects with minimal supervision and to collaborate effectively with faculty, staff, and trainees
- Upload cover letter, resume, and contact information for three professional references
