Glean is looking to improve its AI-powered knowledge management platform by enhancing its document indexing, retrieval, and ranking infrastructure, as well as its generative AI capabilities, to boost user productivity and collaboration.
Requirements
- Strong in statistics and/or machine learning.
- Experience in applying statistical/ML skills into tangible improvements in products, internal tools, and processes.
- Very proficient in Python.
- Proficient in SQL and the modern data stack (e.g. source-controlled dbt pipelines for ETL/ELT).
- Strong at defining good product KPIs/guardrail metrics, dashboarding and analysis of raw data to derive strategic insights.
- Experience working on ranking, developing, and maintaining A/B experimentation platforms and/or ML measurement problems.
Responsibilities
- Define KPIs, create data foundations and build visualizations to measure the efficacy of Glean’s world-leading document indexing, retrieval, ranking infrastructure, which powers its search and LLM-powered products, as well as other ways of content personalization such as ranking agents in an agent library.
- Conduct rigorous empirical analyses rooted in sophisticated techniques and a deep understanding of our ranking stack to identify & prototype ways to improve ranking at Glean.
- Collaborate with product data science and engineering teams to identify techniques, tooling and process improvements in online A/B experimentation to assist rigorous decision making across all relevant product domains.
- Develop and maintain our A/B experimentation platform based on stakeholder feedback
- Write code or identify vendors to deploy these techniques to production in a scalable manner that’s easy to use by engineering, product data science, product management and design teams
- End to end evaluation of various hero use cases like document/URL uploads, including evaluation set generation, coming up with evaluation criteria and methods to interpret the results.
- Breaking down end to end evaluations into more granular evaluation of various tasks & skills including but not limited to content summarization/analysis/generation, multi-step reasoning & strategizing, tool selection & use, coding & system design.
Other
- 5+ years of experience as a Masters degree holder, 3+ as a PhD degree holders (Masters/PhD degree in Statistics, Mathematics or Computer Science, or another quantitative field)
- Pragmatic way that puts business urgencies first.
- Concise and precise in written and verbal communication. Technical documentation is your strong suit.
- Experience in B2B SaaS.
- Passionate about using AI to improve the productivity of data teams as well as non-data professionals trying to derive more value of their company’s data.
- We foster a vibrant company culture through regular events, and provide healthy lunches daily to keep you fueled and focused.
- We are a diverse bunch of people and we want to continue to attract and retain a diverse range of people into our organization. We're committed to an inclusive and diverse company. We do not discriminate based on gender, ethnicity, sexual orientation, religion, civil or family status, age, disability, or race.
