SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Salary: USD 320,000 - 485,000 / annual
Anthropic's Inference team is seeking a Staff or Senior Software Engineer to design, build, and maintain the distributed systems that serve Claude to millions of users worldwide. This role sits at the intersection of infrastructure excellence and AI research enablement.
You will own critical components of Anthropic's inference stack, including intelligent request routing, load balancing, and fleet orchestration across thousands of AI accelerators spanning multiple cloud providers (AWS, GCP, Azure). The team balances two competing mandates: maximizing compute efficiency to support explosive customer growth while providing researchers with high-performance infrastructure to develop next-generation models.
Key responsibilities include:
- Designing and building resilient distributed systems that serve hundreds of thousands of customers daily
- Developing intelligent request routing and load balancing systems across diverse accelerator families
- Building autoscaling and orchestration systems that optimize cost and efficiency across production, research, and experimental workloads
- Creating production-grade deployment pipelines for reliable model releases
- Integrating new AI accelerator platforms and supporting inference for emerging model architectures
- Analyzing observability data to tune performance based on real-world production workloads
- Managing multi-region deployments and geographic routing for global customers
You should have significant experience with distributed systems, ideally including high-performance, large-scale deployments. Preferred experience includes machine learning systems at scale, load balancing/traffic management, LLM inference optimization, Kubernetes, and cloud infrastructure. Proficiency in Python or Rust is valued.
This is a hands-on technical role where you'll work on complex, performance-sensitive systems that directly impact both business results and research breakthroughs. The ideal candidate thrives in environments requiring technical excellence, flexibility, and a bias toward impact.
