College Board is investing in Generative AI ideation through its internal incubator, the GenAI Studio, to responsibly experiment with AI for staff, product development, and educational applications. The role aims to leverage new technologies and data science expertise to solve impactful problems in the education space.
Requirements
- 5+ years' experience in data science, product analytics, or adjacent fields with a focus on applied machine learning
- Experience working with large language models, generative AI APIs, or orchestration frameworks
- Hands-on expertise with Python (e.g., Pandas, LangChain, Scikit-learn) and SQL, as well as familiarity with LLM evaluation tools
- Experience in data and statistical analysis including experience working with and evaluating text data
- Knowledge in developing applications on a Serverless or Microservices cloud platform (AWS/Azure/GCP, etc.)
- Utilize state-of-the-art techniques to evaluate generative model outputs—including off-the-shelf, open-source, and custom models—using quantitative and qualitative methods
- Keep abreast of rapid technological developments and relevant product offerings, especially within the Generative AI space
Responsibilities
- Prototype new tools leveraging emerging technologies and models (50%)
- Collaborate with users, product and strategy teams to scope and define problem statements
- Use orchestration and prompt engineering techniques to build exploratory solutions (e.g., LangChain, LlamaIndex)
- Rapidly prototype user-facing AI tools to test and validate hypotheses
- Create customized or multi-staged models to address nuanced use cases
- Analyze structured and unstructured data to uncover patterns and inform model usage
- Evaluate generative AI models and tools (30%)
Other
- Strong analytical and product thinking skills with a track record of delivering real-world impact
- Excellent communication and listening skills with comfort presenting to both technical and business audiences, including senior leadership
- Eagerness to develop deep understanding of the problems students, educators, and College Board employees face in order to best address them
- Candidates must be authorized to work in the United States for any employer
- Proficiency in Microsoft Suite tools is preferred, though a willingness to learn is equally valued
