Together AI is looking to optimize autoscaling, minimize cold starts, achieve the best end-to-end model performance, and provide a best-in-class developer experience with great tooling for custom models and dedicated inference.
Requirements
- 5+ years of demonstrated experience in building large scale, fault tolerant, distributed systems and API microservices
- Experience running serverless inference platforms, doing model bring-up on short notice, being on call, or general cloud provider is a very big plus
- Experience designing, analyzing and improving efficiency, scalability, and stability of various system resources
- Excellent understanding of low level operating systems concepts including concurrency, networking and storage, performance and scale
- Expert-level programmer in one or more of Golang, Rust, Python, C++, or Haskell
- Proficiency in writing and maintaining Infrastructure as Code (IaC) using tools like Terraform
- Experience with Kubernetes or other container orchestration systems
Responsibilities
- New hires may work on multi-cluster orchestration, portfolio optimization, predictive autoscaling, control panes, model bring-up, light model optimization, APIs for managing deployments, inference worker SDKs, and CLI tools.
- Analyze and improve the robustness and scalability of existing distributed systems, APIs, databases, and infrastructure
- Write clear, well-tested, and maintainable software and IaC for both new and existing systems
- Conduct design and code reviews, create developer documentation, and develop testing strategies for robustness and fault tolerance
- optimizing autoscaling, minimizing cold starts, achieving the best end-to-end model performance, and providing a best-in-class developer experience with great tooling
Other
- Good taste and ability to thoughtfully discuss how what you’ve built has failed over time
- Partner with product teams to understand functional requirements and deliver solutions that meet business needs
- Bachelor’s or Master’s degree in Computer Science, Computer Engineering, or a related technical field, or equivalent practical experience
- Writing-heavy roles or companies are a plus
- competitive compensation, startup equity, health insurance and other competitive benefits
