The Principal Applied Scientist will play a key role in the evaluation of LLM and LMM models, with a focus on creating innovative solutions to real-world problems.
Requirements
- Deep understanding of machine learning techniques, including foundation model evaluation, traditional NLP/CV metrics, LLMaaJ techniques, human evaluation, confidence estimation, agentive application evaluation, and RAG application evaluation.
- Experience in conducting in-depth research
- Producing production-ready code
- Strong background in programming languages, such as Python
- Experience with machine learning frameworks, such as TensorFlow or PyTorch
- Strong understanding of machine learning algorithms and architectures
- Extensive experience in generative AI and model/application evaluation.
Responsibilities
- Conduct in-depth research on foundation model evaluation, including traditional NLP/CV metrics, LLMaaJ techniques, human evaluation, confidence estimation, agentive application evaluation, and RAG application evaluation.
- Produce production-ready code for handoff of POC applications to counterparts in Engineering.
- Work closely with cross-functional teams to integrate evaluation capabilities into various applications and products.
- Identify new opportunities for evaluation and explore emerging technologies.
- Stay Updated: Maintain a deep understanding of industry trends and advancements in evaluation.
- Designing and executing experiments
- Researching new algorithms
Other
- Help lead and mentor a high-performing team of scientists and engineers.
- PhD in Computer Science, Mathematics, Statistics, Physics, Linguistics, or a related field, with a dissertation or thesis centered on machine learning techniques, is preferred, although a Master's or Bachelor's degree with relevant experience will also be considered.
- Strong publication record, including as a lead author or reviewer, in top-tier journals or conferences.
- Excellent problem-solving and analytical skills
- Hiring Range in USD from: $120,100 - $251,600 per year.
