The University of Florida's College of Medicine is seeking a Data Scientist II to lead the analysis and reporting of medical education assessments, with a focus on Clinical Skills Examinations (CSEs), and to support faculty and trainee development through rigorous analytics and scholarly output.
Requirements
- A Bachelor's Degree in data science, statistics, bioinformatics, analytics, or similar field
- Experience with psychometric procedures and methods, including classical test theory, item response theory, and adaptive testing.
- Proficiency in statistical analysis software (e.g., R, Python, SAS, SPSS, Stata)
- Strong skills in data cleaning, management, and quality assurance.
- Experience conducting rigorous analyses in medical education or related social/health sciences
- Experience with technical writing (grants and manuscripts)
- Experience leading standard setting, building weighted grading/score models, and producing longitudinal competency profiles in clinical skills and clerkship contexts.
Responsibilities
- Create psychometric analyses (CTT, IRT, factor, and generalizability/reliability studies) to evaluate and improve high-stakes assessments.
- Conduct inter-rater reliability analyses to improve validity and provide consistency by reducing score drift.
- Collaborate with the Medical Director of ALAC to design and facilitate standard-setting panels, justify defensible passing scores, and revise grading schemas and scoring models.
- Hypothesize ideas to strengthen the reliability, validity, and fairness of CSEs and related high-stakes clinical skills assessments in collaboration with the Medical Director.
- Present actionable psychometric reports to provide continuous quality improvement.
- Lead analysis, interpretation, and reporting of performance for medical education assessments, including performance data for the CSE.
- Develop and maintain learning analytics that inform progress, continuous quality improvement and summative reporting to program leadership.
Other
- A Bachelor's Degree in data science, statistics, bioinformatics, analytics, or similar field and three years of experience; Master's Degree in data science, statistics, bioinformatics, analytics, or similar field and one year of experience; Doctoral Degree in data science, statistics, bioinformatics, analytics, or similar field.
- Ability to manage multiple complex projects with minimal supervision
- Ability to collaborate effectively with faculty, staff, and trainees.
- Strong written and verbal communication for research dissemination
- Must upload cover letter, resume, and contact information for three professional references to apply
