SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Salary: USD 160,900 - 260,700 / annual
Docker is seeking a Senior Software Engineer to design and build the AI Gateway, a control plane for taking AI agents from prototype to production. The Gateway provides model and tool routing with built-in security, an OpenAI-compatible interface, unified observability, and policy enforcement—eliminating the infrastructure toil so developers can focus on building AI agents.
In this role, you will:
- Design and implement core systems including the model router, MCP gateway, and control plane
- Build infrastructure supporting dynamic model selection, auto-failover, cost-based routing, and policy enforcement
- Own critical capabilities such as secure credential storage, session summarization, caching, and rate limiting
- Develop APIs for OpenAI-compatible interfaces and the Model Context Protocol
- Build infrastructure for evaluation, telemetry, replay, and backtesting of agents and LLM workflows
- Lead architectural decisions and mentor engineers as the team scales
- Optimize performance through caching, session summarization, and rate limiting under production load
- Contribute to roadmap planning, technical strategy, and cross-functional alignment
Key problems you'll solve include building unified abstractions over different model and tool providers, implementing secure and fast credential/identity handling for tool access, providing real-time and historical visibility into agent behavior with enforceable policies, and maintaining low latency and high reliability as traffic scales.
Docker is a globally distributed, remote-first team building developer tools trusted by 20+ million monthly users. As AI agents redefine software development, Docker is positioned at the center of that shift, providing sandboxed environments, verified images, and secure infrastructure for trustworthy autonomous workflows.
REQUIREMENTS:
- 6+ years of backend engineering experience with production-grade systems
- Deep knowledge of distributed and highly scalable systems, cloud-native infrastructure, and API design
- Experience building secure, high-throughput services (gateways, proxies, load balancers, policy engines)
- Fluency in Go and/or Rust (both preferred)
- Familiarity with AI/ML platforms or model serving infrastructure
- Strong product mindset and excitement about developer-facing tools
- Ownership mentality with bias for shipping, learning, and iterating
- Bachelor's degree in Computer Science, Engineering, or related field, or equivalent practical experience
BONUS QUALIFICATIONS:
- Prior experience with OpenAI, Anthropic, or similar LLM APIs
- Familiarity with RAG architectures, vector databases, or agent frameworks (LangChain, AutoGPT, CrewAI)
- Experience with policy engines (OPA), observability frameworks (OpenTelemetry), or API gateways
- Understanding of OAuth2.1, secret management, and cloud IAM systems
- Experience with Kubernetes, Docker, and microservices architecture
