SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Luma AI is seeking a Software Engineer to own the inference systems that serve the company's models at scale. You will be responsible for integrating new model architectures into the inference engine, scaling deployments across thousands of machines, and optimizing GPU fleet utilization while maintaining internal SLOs.
This is large-scale systems engineering work focused on model serving infrastructure. You will work on scheduling, fleet management, deployment pipelines, and reliability across clusters and multiple hardware providers. The role is suited for a strong systems engineer comfortable with model serving and Kubernetes at scale. If you prefer pure modeling work, this role is firmly on the systems side.
Key Responsibilities:
- Ship new model architectures by integrating them into the inference engine
- Collaborate across research, engineering, and infrastructure teams to optimize model efficiency and deployments
- Build internal tooling to measure, profile, and track the lifetime of inference jobs and workflows
- Automate, test, and maintain inference services for maximum uptime and reliability
- Manage and optimize inference workloads across clusters and hardware providers, scaling deployments across thousands of machines
- Build scheduling systems that use expensive GPU resources optimally while meeting SLOs
- Maintain CI/CD pipelines for model checkpoints and SDKs
First 90 Days:
Days 1–30: Immerse in the inference stack, learn the fleet architecture, and identify reliability or utilization gaps.
Days 30–60: Integrate a model or ship tooling/scheduling improvements that enhance uptime or GPU utilization.
Days 60–90: Harden deployment pipelines and scheduling across clusters and providers.
Requirements:
- Strong Python and system-architecture skills
- Experience deploying models with PyTorch, Hugging Face, vLLM, SGLang, TensorRT-LLM, or similar frameworks
- Experience with queues, scheduling, traffic control, and fleet management at scale
- Experience with Linux, Docker, and Kubernetes, including orchestration, deployment, and scheduling
- Familiarity with Redis and S3-compatible storage
Nice to Have:
- Modern networking stacks including RDMA (RoCE, InfiniBand, NVLink)
- High-performance large-scale ML systems experience (100+ GPUs)
- CUDA proficiency
- FFmpeg or multimedia processing experience
