SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Salary: USD 155,400 - 233,200 / annual
Harvey is an AI platform transforming legal and professional services by combining frontier agentic AI, enterprise-grade infrastructure, and deep domain expertise. The company is at a critical inflection point with strong product-market fit and world-class investor backing.
You'll own the quality bar for Harvey's human evaluation operations as they scale globally. This role sits at the intersection of Product, Engineering, AI Research, and Applied Legal Research, ensuring that human-generated evaluation data is rigorous, calibrated, and decision-grade before teams rely on it to ship.
Key responsibilities include: defining and maintaining evaluation guidelines, benchmarks, and gold references that contract attorneys work from; standardizing rubric design into repeatable templates and documented methodology; owning contract-attorney quality through onboarding, calibration training, and inter-rater reliability tracking; building and maintaining failure-mode taxonomies that turn raw results into actionable signals; running QA on vendor and contract-attorney deliverables; ensuring quality consistency across jurisdictions and non-English geographies; establishing documented evaluation analysis standards and operational dashboards; and partnering with Applied Legal Research to certify evaluations before scaling.
You'll work alongside the Evaluation Operations Manager (who owns throughput and coordination) and collaborate closely with Applied Legal Researchers, Product, Engineering, and AI Research teams. The role requires setting standards for what "good" looks like, serving as an arbiter on ambiguous judgments, and building stakeholder trust that enables teams to act on evaluation signals with confidence.
The ideal candidate has 6+ years in product operations, research operations, evaluation/QA operations, or quality program management. You should have a proven track record owning quality inputs for complex, expert-driven work; experience onboarding and calibrating distributed pools of expert raters; grounding in measurement concepts like calibration, inter-rater reliability, and rubric design; comfort interpreting evaluation data; and a bias toward scaling and streamlining processes under shipping pressure with strong documentation. You'll translate nuanced domain expertise into repeatable standards and coordinate effectively across functions.
About Harvey
Legal / Compliance / Risk; AI / Data / Infrastructure — AI platform for legal and professional services work.
