SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Salary: USD 201,300 - 352,300 / annual
ServiceNow's AI Services Platform Group is seeking a Senior Staff Machine Learning Engineer (DevOps) to lead the architecture, optimization, and operational excellence of their AI inference platform. This is a high-impact technical leadership role combining deep expertise in GPU compute, LLM APIs, inference frameworks, containerization, and Kubernetes orchestration.
You will own the end-to-end inference lifecycle—from onboarding cutting-edge models to deploying optimized inference engines in production across multiple backends on Kubernetes infrastructure. You'll architect the intelligent gateway layer using Envoy and ext_proc to enforce access control, provide observability, and enable advanced request routing. The role sits at the intersection of ML systems engineering and DevOps, requiring both algorithmic depth and production operational excellence.
Key responsibilities include:
- Designing and operating the AI inference platform serving production workloads
- Deploying, scaling, and operating services in production Kubernetes environments with Helm-based deployments and CI/CD pipelines
- Owning observability end-to-end: meaningful metrics, useful logs, real tracing, and alerts tied to customer impact
- Debugging and resolving production incidents independently, participating in on-call rotations, and running root-cause analysis against defined SLOs
- Collaborating with researchers, applied scientists, and product managers to build AI-powered work experiences
- Using AI coding assistants (GitHub Copilot, Cursor) with critical rigor to review AI-generated code for logic errors, security anti-patterns, and edge cases
The team's dual mission is to build and evolve the AI platform while laying foundations and de-risking emerging AI technologies that unlock new work experiences.
REQUIREMENTS:
- 8+ years of backend software engineering experience
- 4+ years of designing and operating distributed systems
- 3+ years of Kubernetes in production, including Helm and CI/CD
- Expertise in Go or Python, OOP, design patterns, and time/space-efficient algorithms
- Experience building new products using challenging algorithms
- Knowledge of core AI/ML techniques and algorithms
- Knowledge of unit testing, profiling, and code tuning
- Confident use of AI coding assistants with ability to rigorously review AI-generated code
- Bachelor's degree in computer science, software engineering, or closely related technical field (Master's degree may offset up to one year of experience minimum)
- Equivalent practical experience considered where technical depth is clearly demonstrable
