Together AI is looking to scale its data processing components to handle millions of events every day for its mission-critical systems.
Requirements
- Expert-level skills in designing, building, and operating stream processing pipelines with services like AWS Kinesis, Apache Kafka, or Redpanda
- Expert-level knowledge of building real-time customer facing analytics systems using services like AWS TimeStream or Clickhouse
- Proficiency in writing and maintaining Infrastructure as Code (IaC) using tools like Terraform, AWS CDK, or Pulumi
- Proficiency in version control practices and integrating IaC with CI/CD pipelines
- Proficiency in implementing and managing GitOps workflows with tools such as ArgoCD, Github Actions, TeamCity, or similar
- Proficiency in one or more of Golang, Rust, Python, Java, or TypeScript
- Experience with Kubernetes, or containers a plus
Responsibilities
- Identify, design, and develop foundational data infrastructure components capable of handling millions or billions of events daily
- Analyze and improve the robustness and scalability of existing data processing infrastructure
- Write clear, well-tested, and maintainable infra-as-code and software for both new and existing systems
- Conduct design and code reviews, create developer documentation, and develop testing strategies for robustness and fault tolerance
- Partner with product teams to understand functional requirements and deliver solutions that meet business needs
- Participate in an on-call rotation to address critical incidents when necessary
Other
- Bachelor’s or Master’s degree in Computer Science, Computer Engineering, or a related technical field, or equivalent practical experience
- 5+ years of demonstrated experience in building large scale, fault tolerant, distributed data platforms, stream processing pipelines, ETLs, etc
