College Board is looking to solve the problem of applying Generative AI to creatively solve challenging, real-world problems in the education space, specifically in the development of their products and services for students and teachers.
Requirements
- 5+ years' experience in data science, product analytics, or adjacent fields with a focus on applied machine learning
- Experience working with large language models, generative AI APIs, or orchestration frameworks
- Hands-on expertise with Python (e.g., Pandas, LangChain, Scikit-learn) and SQL, as well as familiarity with LLM evaluation tools
- Experience in data and statistical analysis including experience working with and evaluating text data
- Knowledge in developing applications on a Serverless or Microservices cloud platform (AWS/Azure/GCP, etc.)
- Strong analytical and product thinking skills with a track record of delivering real-world impact
- Experience with machine learning, natural language processing, and model evaluation
Responsibilities
- Collaborate with users, product and strategy teams to scope and define problem statements
- Guide product iteration using model results, user feedback, and experimentation insights
- Use orchestration and prompt engineering techniques to build exploratory solutions (e.g., LangChain, LlamaIndex)
- Rapidly prototype user-facing AI tools to test and validate hypotheses
- Create customized or multi-staged models to address nuanced use cases
- Analyze structured and unstructured data to uncover patterns and inform model usage
- Utilize state-of-the-art techniques to evaluate generative model outputs—including off-the-shelf, open-source, and custom models—using quantitative and qualitative methods
Other
- Bachelor’s degree in a quantitative discipline or computer science preferred, Master’s degree a strong plus
- Excellent communication and listening skills with comfort presenting to both technical and business audiences, including senior leadership
- Eagerness to develop deep understanding of the problems students, educators, and College Board employees face in order to best address them
- Clear and concise communication skills, both written and verbal
- Proficiency in Microsoft Suite tools is preferred, though a willingness to learn is equally valued
