SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Salary: USD 201,300 - 352,300 / annual
ServiceNow's AI Services Platform Group is seeking a Senior Staff Machine Learning Engineer (DevOps) to lead the architecture, optimization, and operational excellence of their AI inference platform. This is a high-impact technical leadership role combining deep expertise in GPU compute, LLM APIs, inference frameworks, containerization, and API gateway architecture.
You will own the end-to-end inference lifecycle—from onboarding cutting-edge models to deploying optimized inference engines in production across multiple backends on Kubernetes infrastructure. You'll architect the intelligent gateway layer using Envoy and ext_proc to enforce access control, provide observability, and enable advanced request routing. The role requires pragmatic engineering with measurable performance impact.
Key responsibilities include:
- Designing and operating distributed AI inference systems at scale
- Deploying, scaling, and operating services in production Kubernetes environments with Helm-based deployments and CI/CD pipelines
- Owning observability for systems built—meaningful metrics, useful logs, real tracing, and alerts focused on customer impact
- Debugging and resolving production incidents independently, participating in on-call rotations, and running root-cause analysis against defined SLOs
- Partnering with researchers, applied scientists, and product managers to build AI-powered work experiences
- Leveraging AI coding assistants (GitHub Copilot, Cursor) with critical rigor for code review, catching logic errors, security anti-patterns, and edge cases
This role requires presence in the Santa Clara office two days per week.
REQUIREMENTS:
- 8+ years of backend software engineering experience
- 4+ years of designing and operating distributed systems
- 3+ years of Kubernetes in production, including Helm and CI/CD
- Expertise in Go or Python, OOP, design patterns, and time/space-efficient algorithms
- Experience building new products using challenging algorithms
- Knowledge of core AI/ML techniques and algorithms
- Knowledge of unit testing, profiling, and code tuning
- Confident use of AI coding assistants with ability to rigorously review AI-generated code
- Bachelor's degree in computer science, software engineering, or closely related technical field (Master's degree may offset up to one year of experience minimum)
- Equivalent practical experience considered where technical depth is clearly demonstrable
