SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Anthropic's Inference team is responsible for building and maintaining the critical systems that serve Claude to millions of users worldwide. This Staff Software Engineer role focuses on designing, building, and maintaining the distributed systems infrastructure that powers Claude's inference at scale.
Key responsibilities include:
- Design and build distributed systems serving Claude to millions of users globally
- Develop resilient, flexible systems that adapt in real time to production events
- Develop intelligent request routing, load balancing, and traffic management systems across thousands of accelerators
- Maximize compute efficiency across the fleet through autoscaling and orchestration of production, research, and experimental workloads
- Build and operate production-grade deployment pipelines for releasing new models
- Provide high-performance inference infrastructure enabling researchers to develop next-generation models
- Integrate new AI accelerator platforms and support inference for new model architectures
The team operates under a dual mandate: maximizing compute efficiency to reliably serve explosive customer growth while enabling breakthrough research through high-performance inference infrastructure. You'll tackle complex, distributed systems challenges across multiple accelerator families and emerging AI hardware running in multiple cloud platforms.
Representative projects include designing intelligent routing algorithms, autoscaling compute fleets dynamically, building production deployment pipelines, contributing to new inference features, supporting new model architectures, analyzing observability data for performance tuning, and managing multi-region deployments for global customers.
Required qualifications include significant software engineering experience with distributed systems, results-oriented mindset with bias toward flexibility and impact, willingness to work outside job description, desire to learn about ML systems and infrastructure, and ability to thrive where technical excellence drives business results and research breakthroughs.
Preferred qualifications include experience with high-performance large-scale distributed systems, deploying ML systems at scale, load balancing and traffic management, LLM inference optimization, Kubernetes and cloud infrastructure (AWS, GCP, Azure), and proficiency in Python or Rust.
