SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Judgment Labs is building learning infrastructure for AI agents in production. This role is a full-stack product engineering position focused on designing and building the core platform that enables teams to improve their agents through real production experience.
You will own end-to-end problems spanning product definition, customer collaboration, and implementation. Key areas include:
**Judgment Agent**: Shape how the Judgment Agent runs large-scale investigations across production traces, with parallel investigators examining different dimensions (failure modes, tool errors, regressions, drift) and merging results into actionable insights.
**Verification Platform**: Build hosted simulated environments for stateful agent evaluations, trajectory replay against modified agents, and monitoring systems to catch unintended behavior changes.
**Investigation Interfaces**: Design how engineers understand agent behavior and failures. Create intuitive ways to navigate long traces, tool calls, and decision trees—making thousand-step trajectories legible and debuggable in minutes.
**Swarm UX**: Design interfaces for engineers to monitor and manage hundreds of parallel investigations, redirect investigators when needed, and consume findings efficiently without information overload.
**Improvement Loop**: Build workflows that transform production trajectories into datasets, evaluation judges, and regression checks, creating a seamless path from problem discovery to verified fixes.
**Platform Infrastructure**: Implement workspaces, roles, permissions, billing, usage tracking, and limits for teams running multiple agents across environments.
**SDK & Developer Experience**: Create an SDK and terminal-first experience enabling Claude Code, Codex, and OpenCode sessions to invoke Judgment as a subagent during development.
You'll bring strong full-stack systems thinking, hands-on experience with LLMs or agents, and the ability to work directly with customers to solve real-world problems. High agency, clear communication, and comfort in ambiguous, fast-moving environments are essential.
