Salesforce is seeking a Software Quality Lead Member of Technical Staff to drive the integration of AI/ML solutions, particularly Large Language Models (LLMs), into their products. The role aims to ensure high-quality AI-driven solutions for enterprise applications and to maintain Salesforce's leadership in AI innovation.
Requirements
- AI & Machine Learning Expertise: Strong understanding of AI/ML principles, particularly in evaluating and optimizing model performance, response consistency, and safety.
- LLM Expertise: Hands-on experience evaluating Large Language Models (LLMs) for quality response, including familiarity with GPT-like models and AI evaluation frameworks.
- Focus on Delivery: Ability to deliver practical solutions for an active user base, which they are expected to follow-up with enhancements and improvements, as needed.
- Prior experience in a Data Science or a Machine Learning Science or a Machine Learning Engineering team.
- Experience in ethical AI practices and evaluating models for fairness, bias, and user safety.
- Strong understanding of public cloud infrastructure - AWS/Azure/GCP.
- Can Architect, design, implement, test and deliver test frameworks for highly scalable products.
Responsibilities
- Drive Innovation & Adoption: Identify opportunities for integrating AI/ML solutions into existing workflows, propose new ones that are high-quality and efficient, and generally positively influence engineering decision-making by communicating and evaluating solution options, and facilitating agreement among key stakeholders.
- Continued Excellence in AI/ML: Track publicly available models, frameworks, libraries, and publications, while focusing on their practical application and relevance to Salesforce.
- Technical Leadership on Proof of Concepts: Influence the direction of R&D through technical, process, or product knowledge leadership.
- Collaborate on AI Model Improvements: Work closely with data scientists and machine learning engineers to provide feedback on LLM performance and recommend improvements based on quality testing results.
- Evaluate LLM Performance: Assess and optimize Large Language Models (LLMs) for quality, accuracy, relevance, and safety in various use cases.
- Human Evaluation: Oversee human evaluation processes for subjective quality dimensions, such as response engagement, user safety, and contextual accuracy.
- Define Quality Strategies: Develop and implement robust quality strategies for Salesforce’s SaaS products, focusing on AI/ML and LLM integrations.
Other
- Excellent communication skills, with the ability to work cross-functionally with product managers, AI/ML engineers, and data scientists.
- Strong analytical and problem-solving abilities with a focus on identifying quality gaps and driving improvement.
- Bachelor’s degree or higher (implied by the context of the role).
- Willingness to collaborate and work in a fast-paced, innovative environment.
- Certification in Salesforce.
