SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Pluralis Research is building Protocol Learning: a system for training and serving large language models in a fully decentralized way on consumer-grade devices connected via the internet. The company has already demonstrated feasibility with Agora, a permissionless run that pretrained an 8B model from scratch on consumer GPUs distributed across the internet, with no single participant ever holding the full model weights.
As a Research Scientist focused on pre-training, you will build the training infrastructure that scales Protocol Learning from the 8B proof-of-concept to frontier-scale models. This role bridges research and systems engineering: you'll implement and optimize distributed training across heterogeneous hardware, different geographic regions, and ordinary internet connections—an environment that breaks nearly every assumption of traditional datacenter training.
Key responsibilities include:
- Distributed pretraining: Implement and optimize model-parallel training using data, pipeline, and tensor parallelism for large models on heterogeneous GPUs under low-bandwidth, high-latency links.
- Performance optimization: Develop techniques that reduce communication overhead while maintaining model convergence in challenging network environments.
- Elasticity and fault tolerance: Design robust checkpointing, state synchronization, and recovery mechanisms as participants join and leave the network mid-run.
- Run instrumentation: Build monitoring systems that track throughput, bottlenecks, and model quality across hundreds of distributed devices.
The company's published research includes Subspace Networks, Factored Gossip DiLoCo, AsyncMesh, and Sentinel—all addressing the unique challenges of decentralized training.
Pluralis is backed by Union Square Ventures and other tier-1 investors, with a world-class team of ML researchers distributed globally across Australia and North America. The team is deeply technical and ideologically committed to implementing a protocol for intelligence that is collective, trustless, and sovereign.
REQUIREMENTS:
- Hands-on distributed training (required): Proven experience training models across many devices in PyTorch using FSDP, DeepSpeed, Megatron, or custom implementations. Strong understanding of data, tensor, and pipeline parallelism.
- Strong engineering: Production-quality Python code, concurrency, failure handling, and profiling expertise.
- Evidence of execution: Shipped systems, research code, open-source contributions, or serious personal projects demonstrating technical depth.
- Mission alignment: Belief that Protocol Learning represents a viable third path for collective, trustless, and sovereign AI.
NICE TO HAVE:
- Hands-on experience training or serving large language models (Nemotron, Qwen, OLMo, or similar).
- P2P networking and NAT traversal experience.
- Post-training and reinforcement learning experience.
- Inference and serving systems experience.
- Background at proprietary, open-weight, or open-source AI labs.
- Professional-level English proficiency (written and spoken) required for remote global collaboration.
