Filtr uses AI to scan 1000+ jobs and finds postings that perfectly matches your resume

Anthropic's Inference team is responsible for building and maintaining the critical systems that serve Claude to millions of users worldwide, aiming to maximize compute efficiency for explosive customer growth and enable breakthrough research by providing high-performance inference infrastructure.

Requirements

• significant software engineering experience, particularly with distributed systems
• High-performance, large-scale distributed systems
• Implementing and deploying machine learning systems at scale
• Load balancing, request routing, or traffic management systems
• LLM inference optimization, batching, and caching strategies
• Kubernetes and cloud infrastructure (AWS, GCP)
• Python or Rust

Responsibilities

• building and maintaining the critical systems that serve Claude to millions of users worldwide
• serving our models via the industry's largest compute-agnostic inference deployments
• responsible for the entire stack from intelligent request routing to fleet-wide orchestration across diverse AI accelerators
• tackle complex, distributed systems challenges across multiple accelerator families and emerging AI hardware running in multiple cloud platforms
• Designing intelligent routing algorithms that optimize request distribution across thousands of accelerators
• Autoscaling our compute fleet to dynamically match supply with demand across production, research, and experimental workloads
• Building production-grade deployment pipelines for releasing new models to millions of users

Other

• results-oriented, with a bias towards flexibility and impact
• Pick up slack, even if it goes outside your job description
• Enjoy pair programming (we love to pair!)
• Want to learn more about machine learning systems and infrastructure
• Thrive in environments where technical excellence directly drives both business results and research breakthroughs