SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Salary: USD 201,300 - 352,300 / annual
ServiceNow's AI Services Platform Group is seeking a Senior Staff Machine Learning Engineer (DevOps) to lead the architecture, optimization, and operational excellence of their AI inference platform. This role combines deep technical expertise in GPU compute, LLM APIs, inference frameworks, containerization, orchestration, and API gateway architecture with pragmatic engineering and measurable performance impact.
You will own the end-to-end inference lifecycle—from onboarding cutting-edge models to deploying optimized inference engines in production across multiple backends on Kubernetes infrastructure. You'll architect the intelligent gateway layer using Envoy and ext_proc to enforce access control, provide observability, and enable advanced request routing. The role requires independent debugging and resolution of production incidents, participation in on-call rotations, and root-cause analysis against defined service level objectives.
The AI Services Platform Group is a customer-focused innovation group building intelligent software and smart user experiences. The team comprises researchers, applied scientists, engineers, and product managers with a dual mission: building and evolving the AI platform while partnering with teams to build AI-powered work experiences, and laying foundations to research and de-risk emerging AI technologies.
This is a hybrid role requiring presence in the Santa Clara office two days per week.
REQUIREMENTS:
- 8+ years of backend software engineering experience
- 4+ years of designing and operating distributed systems
- 3+ years of Kubernetes in production, including Helm and CI/CD
- Expertise in Go or Python, OOP, design patterns, and time/space-efficient algorithms
- Experience building products using challenging algorithms
- Knowledge of core AI/ML techniques and algorithms
- Knowledge of unit testing, profiling, and code tuning
- Ability to deploy, scale, and operate services in production Kubernetes environments
- Ownership of observability: meaningful metrics, useful logs, real tracing, and alerts
- Confident use of AI coding assistants (GitHub Copilot, Cursor) with critical review capability
- Understanding of data privacy rules governing prompt engineering
- Bachelor's degree in computer science, software engineering, or closely related technical field (master's degree may offset up to one year of experience requirement)
- Equivalent practical experience considered where technical depth is clearly demonstrable
