SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Turing builds infrastructure that frontier AI labs use to evaluate coding agents and language models. This role owns features across the full stack—from data models and background jobs to user-facing interfaces—for benchmark datasets, agent evaluation runs at scale, and automated task quality checks.
You will design and ship end-to-end features: data models, APIs, background workers, and UI when needed. Key responsibilities include building and improving how agents and models are evaluated by running trials in sandboxed environments, scoring results, and aggregating thousands of runs into trustworthy results. You'll improve benchmark task quality-checking through verifiers, rubric-based and LLM-as-a-judge grading, and determining whether tasks are clear, solvable, and fair. You'll integrate new models, agents, and benchmarks in generalizable ways without special cases. You'll keep the system fast and reliable as usage grows through efficient queries, safe migrations, and concurrency that handles load. You'll also review teammates' pull requests and help maintain codebase consistency.
Turing's mission is to accelerate superintelligence by working with frontier AI labs to generate high-quality datasets, reinforcement learning environments, and research benchmarks. The company also works with Fortune 500 enterprises across financial services, life sciences, healthcare, retail, automotive, and CPG to build and deploy agentic AI systems in mission-critical workflows. Turing closes the loop between frontier research and enterprise deployment, turning real-world signals into better data, evaluations, and more capable models.
The team values written engineering guidelines applied consistently, business logic in services, hot paths kept cheap, and simple designs over clever ones. A strong first month includes several merged changes with light review and ownership of one system area you can change confidently.
REQUIREMENTS:
- Typically 5+ years building and operating production backend systems, with ownership of at least one service from design through on-call
- Strong Python; experience with Django (or similar framework), PostgreSQL, and a task queue such as Celery
- Hands-on experience in LLM evaluation beyond calling a model API; understanding of how benchmark tasks are built, how agent trials are scored, and how results are aggregated and compared; experience with Harbor, Terminal-Bench, SWE-bench, Inspect, or similar harness is a strong plus
- Judgment about task quality: ability to spot ambiguous instructions, weak or gameable verifiers, leaked answers, and checks that pass for the wrong reason
- Good tradeoffs: reaching for the smallest design that solves the problem, extending what exists before adding new, and explaining what you chose not to build
- Ownership: taking fuzzy requests, finding relevant code, asking questions only when answers change design, and finishing jobs including migrations, tests, and edge cases
- Quick adoption of team conventions; reading written guidelines and matching them in naming, structure, tests, and error handling from first pull requests
- Comfort with TypeScript and React frontend when changes are visible there (not required to be a specialist)
- Clear writing: pull request descriptions and design notes that explain why, not just what
NICE TO HAVE:
- Experience running agents in sandboxed or containerized environments (Docker, Kubernetes, Modal, Daytona)
- Experience with LLM-as-a-judge or rubric-based grading and reducing judge bias and noise
- Familiarity with coding agents such as Claude Code, Codex, or OpenHands, and with MCP
- Experience with distributed job scheduling, concurrency limits, or database locking at scale
- Contributions to open-source evaluation or infrastructure projects
