LMArena is seeking a Senior Machine Learning Engineer to help scale and strengthen the core infrastructure that powers real-world AI evaluation. This role will focus on building, deploying, and improving model benchmarking systems, addressing challenges in data pipelines, inference APIs, and developing new evaluation methodologies to assess cutting-edge AI.
Requirements
- Strong programming skills with the ability to work across the stack in a typical recommendation system or LLM stack
- Experience in deep learning, language models or reward model training
- Experience in working with LLM for fine tuning, prompt engineering, function calling etc
- Solid understanding of statistics, and various tools and methodologies for evaluating uncertainty in a way that is specific to the given product being shipped
Responsibilities
- Architect and build what will become our core modeling for data and evaluation products
- Own the full stack data, model training, and eval pipelines
- Conduct research into state-of-the-art evaluation methods and contribute to the long-term vision for a centralized, scalable evaluation platform.
Other
- Help grow a culture of feedback and rapid product iteration as we build new features as a tight-nit team
- Self-motivated with a willingness to take ownership of tasks
- A passion for shipping quality products
- 4+ years of industry experience or relevant projects
