SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Salary: USD 320,000 - 485,000 / annual
Anthropic's Inference team is seeking a Staff or Senior Software Engineer to design, build, and scale the critical distributed systems that serve Claude to millions of users worldwide. This role sits at the intersection of infrastructure excellence and AI research enablement.
You will own the full stack of inference infrastructure, from intelligent request routing and load balancing across thousands of accelerators to fleet-wide orchestration across multiple cloud providers (AWS, GCP, Azure). Key responsibilities include:
- Designing and maintaining distributed systems serving Claude at massive scale (hundreds of thousands of customers daily)
- Developing resilient, adaptive systems that respond to real-world events in real time
- Building intelligent request routing, load balancing, and traffic management across diverse AI accelerators and cloud platforms
- Optimizing compute efficiency and cost through autoscaling and workload orchestration across production, research, and experimental environments
- Creating production-grade deployment pipelines for reliable model releases to users
- Providing high-performance inference infrastructure that enables researchers to develop next-generation models
- Integrating new AI accelerator platforms and supporting inference for emerging model architectures
Representative projects include designing routing algorithms for multi-accelerator environments, building dynamic autoscaling systems, creating deployment pipelines for millions of users, analyzing observability data to tune performance, and managing multi-region deployments for global customers.
Anthropnic is a public benefit corporation headquartered in San Francisco, focused on creating reliable, interpretable, and steerable AI systems. The team values impact, technical excellence, and collaborative research culture.
REQUIREMENTS:
- Significant software engineering experience, particularly with distributed systems
- Results-oriented mindset with bias toward flexibility and impact
- Willingness to work outside job description as needed
- Desire to learn about machine learning systems and infrastructure
- Bachelor's degree or equivalent combination of education, training, and professional experience in a field relevant to the role
- Years of experience will correlate with internal job level requirements
PREFERRED QUALIFICATIONS:
- Experience with high-performance, large-scale distributed systems
- Experience implementing and deploying machine learning systems at scale
- Experience with load balancing, request routing, or traffic management systems
- Familiarity with LLM inference optimization, batching, and caching strategies
- Experience with Kubernetes and cloud infrastructure (AWS, GCP, Azure)
- Proficiency in Python or Rust
