SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Datadog's Deployment Gates team is hiring a Senior Software Engineer to build AI-driven systems that decide whether software should ship to production. You'll work on an analysis service that prevents incidents by detecting faulty software changes before they reach customers, sitting directly in CI/CD pipelines.
In this role, you will design and ship decision systems that evaluate customer deployments using CI/CD context and Datadog telemetry, producing safe, explainable allow/block outcomes. You'll own the full lifecycle: defining precision, recall, and trust metrics; building offline and online evaluations; validating changes in shadow mode; and safely promoting improvements to enforcement.
Key responsibilities include designing for robustness and safety by implementing conservative defaults, guardrails, fallbacks, and human-in-the-loop paths so gates behave predictably under noisy or incomplete data. You'll partner closely with Product to translate customer problems and roadmap goals into concrete technical decisions, and integrate across the Datadog platform with internal AI teams building faulty deployment detection pipelines and LLM/agent systems.
You'll own production systems—building and operating reliable backend services that run in the critical path of customer deployments, with on-call responsibilities. This is a highly product-minded engineering role spanning problem discovery, UX, and reliable, scalable production systems.
Required: 5+ years backend systems and microservices experience (tracing, latency, concurrency, resiliency); proficiency in modern programming languages; strong API/service design and production ops skills; proven experience delivering LLM/agent features to production; comfort owning user journeys from prototype to GA; end-to-end AI implementation ownership; fluency with offline/online evals; and demonstrated ability to use AI coding tools effectively.
Bonus: experience with continuous delivery tools (ArgoCD, Argo Rollouts, Spinnaker), planning/agent frameworks, tool-use orchestration, RAG, and retrieval/indexing for observability data.
