SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Mercor is a Series C AI data company ($10B valuation) building infrastructure to measure and improve AI agents in enterprise settings. The company operates a platform where millions of domain experts train frontier AI models, and now brings this expertise-capture capability to Fortune 500 companies through enterprise agents.
As a Member of Technical Staff on the Enterprise Evals Platform, you will own the evaluation and measurement systems that determine whether AI agents are reliable and economically viable before deployment. This is a hybrid research and platform engineering role focused on building the infrastructure that makes agent evaluation scalable, rigorous, and production-ready.
Key responsibilities include: decomposing real-world tasks and encoding expert quality standards into measurable rubrics (golden sets); building verifiers that assess agent trajectories and outputs in ways that are hard to game; constructing the eval platform that runs offline environments, task suites, and grading infrastructure at scale; analyzing production failures to create regression tests; optimizing across models, prompts, and agent configurations; and owning the rollout gates that gate agent changes to customers.
You will partner closely with the Enterprise Platform team and applied AI engineers embedded with customers, giving you visibility into how evals impact real deployments. The role requires deep understanding of agent runtimes, evaluation methodologies (familiarity with benchmarks like APEX, terminal-bench, tau-bench), and the ability to translate fuzzy notions of quality into measurable, defensible standards. Strong software engineering fundamentals and comfort with ambiguous, loosely specified problems are essential.
Mercor operates in-person five days per week from San Francisco, NYC, or London offices. The company is profitable and well-funded, offering competitive equity, relocation support, housing bonuses, and comprehensive benefits.
