Apple is looking to define how to measure, monitor, and improve the performance of AI systems that power next-generation user experiences by developing robust evaluation frameworks and practical tooling for Generative AI applications.
Requirements
- Programming skills in Python
- Experience developing scalable and robust services with FastAPI or similar frameworks.
- Experience in Machine Learning, with a particular emphasis on Large Language Models (LLMs), Retrieval Augmented Generation (RAG) or GenAI Agents
- Experience with evaluating and optimizing Generative AI platforms or applications
- Experience with GenAI RAG and Agent evaluation frameworks like RAGAS, DeepEvals, OpenEvals, AgentEvals or OpenAI Evals
- Familiarity with LLM Observability techniques and best practices
- Experience with cloud platforms like AWS, GCP, or Azure
- Knowledge of containerization and orchestration tools like Docker and Kubernetes
Responsibilities
- Designing and developing platform features for helping solution developers to experiment and identify optimal configurations for delivering high quality GenAI applications.
- Evaluating and analyzing the performance of GenAI applications, and actively collaborating with the team in driving performance improvements
- Translating the latest research into reliable and scalable evaluations that can deliver high quality experiences for our users.
- Actively engaging in all aspects of feature development, from ideation and experimentation to deployment and maintenance.
- Communicating complex technical topics effectively to a diverse audience.
Other
- Bachelor's in Computer Science, Artificial Intelligence, Machine Learning, or a related field or experience
- 2+ years of software engineering experience
- Proven ability to comprehend, interpret, and apply cutting-edge research into tangible applications
- Proven problem-solving and leadership abilities, with the capacity to steer the team's research and build practical applications in a collaborative and fast-paced environment
- Customer-focused with strong business acumen, capable of translating business needs into impactful technical solutions and a proven history of successfully shipping products that drive significant outcomes
- Creative, collaborative and project focused with an ability to work hands-on in multi-functional teams
- Excellent communication skills with the ability to communicate with all stakeholders effectively, including senior leadership
- Master's in Computer Science, Artificial Intelligence, Machine Learning, or a related field
