SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
OpenRouter is the leading AI routing and infrastructure layer that enterprises use to access, manage, and optimize large language models across 80+ providers without lock-in or unnecessary cost. The company routes nearly a billion requests and over 20 trillion tokens daily, serving the most advanced AI teams globally.
You will be OpenRouter's first AI Inference SRE, owning the operational health of the provider supply chain. Your mission is to ensure every endpoint is fast, correct, and available—detecting and routing around problems before customers experience impact.
Key responsibilities include:
- Build and own comprehensive monitoring for every provider and endpoint, tracking latency, throughput, error rates, uptime, and output correctness. Define SLOs per provider tier and establish actionable alerting.
- Improve detection of degraded endpoints and collaborate with the routing team to automatically shift traffic away from problematic providers.
- Own on-call for provider incidents: triage issues, mitigate problems, communicate with external providers, run postmortems, and drive follow-ups to closure.
- Translate telemetry into provider scorecards and SLO reporting that drives accountability and serve as the technical escalation point for misbehaving endpoints.
- Build continuous canaries and evals to catch silent regressions (quantization changes, broken tool calling, truncated streams, pricing mismatches) beyond simple downtime detection.
- Automate manual provider-ops work—disabling endpoints, capacity changes, deprecations, rate-limit tuning—with safe, auditable tooling.
- Develop load-testing tooling to validate endpoint readiness before major launches, preventing day-zero traffic incidents.
You'll report to the Provider Operations Manager and work closely with routing and infrastructure teams.
REQUIREMENTS:
- 4+ years in SRE, production engineering, or infrastructure roles managing high-traffic, customer-facing systems
- Strong expertise with observability tooling: metrics, tracing, logs, SLOs/error budgets, and actionable alerting
- Capable software engineer who prefers building tools over executing runbooks; proficiency in TypeScript and/or Python
- Deep understanding of distributed systems failure modes: timeouts, retries, backpressure, partial outages, noisy neighbors
- Calm, clear incident commander with strong communication skills, especially under pressure with external partners
- Understanding of or eagerness to learn LLM inference serving: streaming, tool calling, prompt caching, throughput/latency tradeoffs, and provider API differences
NICE TO HAVE:
- Experience at an inference provider, model lab, GPU cloud, or API gateway/CDN company
- Familiarity with TypeScript, Cloudflare Workers, Postgres, ClickHouse, GCP, or Vercel
- Background in routing, load balancing, or traffic management systems
- Experience with evals or synthetic monitoring for ML systems
