SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Luma AI is seeking a Research Scientist / Engineer to build and optimize the distributed systems that train large-scale multimodal models across thousands of GPUs. This role focuses on enabling researchers to innovate on top of reliable, efficient, and scalable infrastructure.
You will own the design, implementation, and optimization of distributed training systems for foundation models at massive scale. This is deep PyTorch, CUDA, and distributed-systems work involving advanced parallelism techniques, training stability optimization, and resource utilization across GPU clusters. The role requires hands-on expertise solving real problems in foundation-model training at scale.
Key Responsibilities:
- Design, implement, and optimize efficient distributed training systems for models across thousands of GPUs
- Research and implement advanced parallelization strategies (FSDP, Tensor Parallel, Pipeline Parallel, Expert Parallel)
- Build monitoring, visualization, and debugging tools for large-scale training runs
- Optimize training stability, convergence, and resource utilization across massive clusters
First 90 Days (Illustrative):
- Days 1–30: Immerse in the current training stack and diagnose stability and utilization pain points at scale
- Days 30–60: Ship and validate a parallelization or stability improvement that measurably helps a real training run
- Days 60–90: Build monitoring and tooling that keeps large runs reliable and efficient
About Luma: Luma's mission is to build unified general intelligence that can generate, understand, and operate in the physical world. The company believes multimodality is critical for intelligence, with vision as the next step beyond language models.
Requirements:
- Extensive distributed PyTorch training experience and deep knowledge of parallelism techniques in foundation-model training
- Deep understanding of GPU clusters, networking, and storage systems
- Familiarity with communication libraries (NCCL, MPI) and distributed-system optimization
- Prior experience at the level of FSDP and multi-node training is essential
Nice to Have:
- Strong Linux systems administration and scripting skills
- Experience managing training runs across 100+ GPUs
- Experience with containerization, orchestration, and cloud infrastructure
