Truveta is seeking to evaluate the effectiveness of its AI models utilizing Large Language Models (LLMs) to achieve its vision of Saving Lives with Data
Requirements
- Master's or Ph.D. in Computer Science, Machine Learning, Artificial Intelligence, or a related field
- 7+ years of experience in machine learning, with a focus on natural language processing and large language models
- Proven track record of evaluating and improving AI models, particularly those involving LLMs
- Strong programming skills in Python and experience with machine learning frameworks such as TensorFlow, PyTorch, or similar
- Familiarity with performance evaluation metrics and techniques for LLMs
- Experience with popular LLMs such as GPT, BERT, T5, or similar
- Knowledge of deployment and scaling of AI models in a production environment
Responsibilities
- Evaluate the performance of AI models that leverage Large Language Models (LLMs) in various applications
- Develop and implement metrics to measure the accuracy, efficiency, and overall effectiveness of LLM-based AI models
- Conduct thorough analyses to identify strengths and weaknesses in model performance
- Provide detailed reports and recommendations to improve model accuracy, efficiency, and scalability
- Collaborate with the development team to refine and enhance AI models based on evaluation results
- Stay up-to-date with the latest advancements in machine learning, NLP, and LLMs to ensure our models remain at the forefront of the industry
- Ensure that models meet business requirements and deliver tangible value to our clients
Other
- Master's or Ph.D. in Computer Science, Machine Learning, Artificial Intelligence, or a related field
- 7+ years of experience in machine learning
- Ability to communicate complex technical concepts to non-technical stakeholders
- In person attendance is required for two weeks during the year for Truveta Planning Week
- Must be based in the US and eligible to work
