The company is looking to optimize machine learning inference solutions for both Tensor Processing Units (TPUs) and Graphics Processing Units (GPUs) to provide a seamless, efficient, and cost-effective inference experience for users.
Requirements
- 5 years of experience with machine learning frameworks (e.g., PyTorch, JAX)
- 3 years of experience with ML inference concepts and optimization techniques
- Experience with cloud platforms (e.g., Google Cloud Platform) and open-source development and community collaboration
- Experience with TPU architecture and programming
- Understanding of distributed systems and technologies such as disaggregated serving, speculative decoding and inference solution performance benchmarking and optimization
- Experience contributing to open-source projects particularly in ML or infrastructure
- Experience with software design and architecture
Responsibilities
- Provide technical leadership on projects
- Develop and optimize ML inference solutions for both TPUs and GPUs
- Contribute to internal and Google Cloud Platform (GCP)-specific repositories for inference optimization and integration with cloud services
- Work on optimizing model serving infrastructure and performance focusing on cost efficiency
- Debug and troubleshoot inference service issues developing debugging tools
- Implement TPU-specific backends for virtual large language models (vLLM) and related projects
- Participate in community engagement and governance for open-source repositories
Other
- Bachelor’s degree or equivalent practical experience
- 8 years of experience in software development
- Master’s degree or PhD in Engineering, Computer Science, or a related technical field (preferred)
- 3 years of experience in a technical leadership role leading project teams and setting technical direction (preferred)
- 3 years of experience working in an organization involving cross-functional or cross-business projects (preferred)
- Collaborate with a cross-functional team of engineers, engineering managers and product managers to achieve ambitious goals
