Pluralis Research is working on Protocol Learning, a method for training large models in a fully decentralized way on small consumer-grade devices. The business problem is to overcome the perceived infeasibility of this approach and unlock new challenges in truly open innovation for large models, preventing corporations from monopolizing model development, access, and release.
Requirements
- Strong programming abilities in PyTorch.
- Experience training models across multiple devices.
- Exposure to distributed training, or federated learning.
- Experience with large language models.
- Experience with post training.
- Experience with RL.
Responsibilities
- Solve hard research problems in Protocol Learning.
- Publish foundational papers in Tier-1 Venues.
- Train large models in a fully decentralised way on small consumer-grade devices connected via the internet.
- Train an 8B llama model from scratch with devices in physically different regions.
- Work at the intersection of AI and distributed systems.
- Tackle challenging research problems in the large model regime.
Other
- PhD in Machine Learning with publications in top-tier conferences (NeurIPS, ICML, ICLR).
- Visa sponsorship and relocation support to either US or Australia.
- Remote-First Culture: Flexible work environment with team members distributed globally.
- Professional-level English proficiency (written and spoken).
- Comfortable working across timezones and collaborating with a diverse, distributed group.
