SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Salary: USD 305,000 - 385,000 / annual
Anthropic is hiring a Product Designer for Evals & Prompts to join the Product Prompt and Eval Design team. This role is foundational to building evaluation systems that measure and validate Claude's behavior across product surfaces.
Key responsibilities include:
- Writing and revising prompts behind Claude's tools, features, and behaviors; testing surfaces and shipping prompt fixes
- Building automated graders that validate prompt fixes and rerun across model versions
- Creating low-code, visual eval tools that enable non-technical designers to assemble comparison sets, define rubrics, and compare prompt variants without engineering support
- Observing designer workflows and iterating on tool usability
- Supporting model releases by testing surfaces, writing prompt migrations, and providing data-driven recommendations
- Standing up and scaling the eval harness to exercise 50-100 tools with reproducible settings across model versions
- Packaging evaluation findings for model training, including graders for crisp behaviors and human-feedback signals for fuzzy attributes
Minimum qualifications:
- Production-quality Python
- Experience building and maintaining LLM evaluation pipelines (graders, rubrics, comparison sets, regression suites)
- Experience building internal tools with real interfaces for non-technical users
- Experience with test harnesses, tool-call sandboxing, and reproducible test settings
- Shipping prompts or close collaboration with prompt engineers; understanding cross-model prompt degradation
- Ability to read and analyze transcripts, not just interpret scores
Preferred qualifications include model-launch cycle experience, A/B testing and offline-to-online outcome correlation, front-end or notebook-to-app development, turning product rubrics into training signals, and a user-centric perspective on model behavior.
