xAI is looking to solve the problem of building petabyte-scale, high-throughput data processing systems for AI training.
Requirements
- Strong engineering skills with Python, JAX, Rust, and Spark
- Experience building bespoke data processing libraries from scratch
- Designing and implementing distributed systems in Rust
- Keeping up with state-of-the-art techniques for preparing AI training data
- Organizing and meticulously bookkeeping data across multiple clouds, of multiple modalities, and from many sources
Responsibilities
- Building petabyte-scale, high-throughput data processing systems
- Managing workloads across large cloud compute clusters
- Pre-processing datasets for AI training
Other
- Strong communication skills
- Ability to concisely and accurately share knowledge with teammates
- Work ethic and strong prioritization skills
- Located near the Bay Area or open to relocation
- Annual Salary Range: $180,000 - $440,000 USD
