SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Pluralis Research is building Protocol Learning: a system for training and serving large language models in a fully decentralized way on consumer-grade devices connected via the internet. The company has already demonstrated feasibility with Agora, a permissionless run that pretrained an 8B model from scratch on consumer GPUs distributed across the internet, with no single participant ever holding the full model weights.
As a Research Scientist focused on pre-training, you will build the training infrastructure that scales Protocol Learning from 8B to frontier-scale models. This involves solving novel distributed systems challenges that break traditional datacenter training assumptions: communication-efficient training across heterogeneous hardware and networks, fault tolerance as nodes join and drop mid-run, and robustness to malicious participants.
Key responsibilities include:
- Implementing and optimizing model-parallel training (data, pipeline, and tensor parallelism) for large models on heterogeneous GPUs under low-bandwidth, high-latency internet links
- Developing performance optimization techniques that reduce communication overhead while maintaining model convergence in challenging network environments
- Building elasticity and fault tolerance mechanisms: robust checkpointing, state synchronization, and recovery as participants join and leave the network
- Creating run instrumentation and monitoring systems that provide visibility into throughput, bottlenecks, and model quality across hundreds of distributed devices
The role is deeply technical and research-forward. You'll be working on problems that largely have no published solutions yet, contributing to foundational work in decentralized AI training.
The company is backed by Union Square Ventures and other tier-1 investors, with a world-class team of ML researchers distributed globally across Australia and North America. The culture is remote-first with flexible work arrangements.
REQUIREMENTS:
- Hands-on distributed training experience (required): You must have trained models across many devices in PyTorch using FSDP, DeepSpeed, Megatron, or your own implementation. Deep understanding of data, tensor, and pipeline parallelism is essential.
- Strong software engineering: Production-quality Python code, concurrency, failure handling, and profiling expertise.
- Evidence of execution: Shipped systems, research code, open-source contributions, or serious personal projects demonstrating technical depth.
- Mission alignment: Belief that Protocol Learning is a viable third path for collective, trustless, and sovereign AI.
NICE TO HAVE:
- Hands-on experience training or serving large language models (Nemotron, Qwen, OLMo, etc.)
- P2P networking and NAT traversal experience
- Post-training and reinforcement learning experience
- Inference and serving systems experience
- Background at proprietary, open-weight, or open-source AI labs
ADDITIONAL NOTES:
- Visa sponsorship and relocation support available to Australia or the US
- Professional-level English proficiency (written and spoken) required
- Comfort working across timezones essential given global team distribution
