SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Salary: GBP 90,630 - 122,617 / annual
Duolingo is seeking a Proctor Quality and Learning Manager to own the end-to-end quality standard and learning system for the Duolingo English Test (DET) global proctor network. This is an individual-contributor role with functional leadership over Training Specialists and adjudicators.
You will be responsible for maintaining the DET proctoring quality standard by developing and updating rubrics, decision rules, examples, and guidance that define accurate decisions across routine and difficult edge cases. You'll lead calibration and adjudication cycles, operating recurring Proctor Certification Tests, resolving disputed decisions, identifying disagreement patterns, and ensuring consistent, auditable outcomes.
You will build and run an ongoing learning program, publishing a forward-looking training calendar, delivering monthly interventions, and launching targeted training in response to quality signals, threats, or workflow changes. Using calibration results, audits, and performance trends, you'll measure training effectiveness and identify when to clarify standards, retrain, or update guidance.
You'll guide the execution of Training Specialists and adjudicators by setting expectations, assigning deliverables, reviewing outputs, and operating onboarding and certification programs that ramp new specialists quickly. You'll maintain durable sources of truth through current calibration, adjudication, and training runbooks to ensure critical processes are documented and auditable.
As security threats and technology evolve, you'll translate new risks and workflow changes into clear standards, training, and quality controls in partnership with Psychometrics, Security, Operations, Product, and Engineering teams. You'll represent the proctoring craft by bringing frontline proctor evidence into product experiments and assessing whether launched changes preserve decision quality. You'll also establish benchmarks for AI-assisted workflows and validate automated capabilities against real sessions before deployment.
REQUIREMENTS:
- 5+ years in assessment development, scoring/rater quality, certification, quality assurance, or a field involving complex, consequential judgments; experience aligning reviewers to a shared standard through calibration programs
- Adult-learning program experience spanning onboarding, certification, ongoing training, remediation, and effectiveness measurement (beyond one-off materials)
- Exceptional attention to detail and strong judgment in ambiguous cases; ability to produce clear, defensible decisions and guidance
- Analytical fluency to interpret quality data, disagreement patterns, and performance trends and translate them into practical interventions
- Clear written and verbal communication; ability to turn complex standards into precise, usable guidance and explain difficult calls
- Cross-functional leadership without formal authority; experience directing contractors or partners toward clear accountability and outcomes
- Demonstrated skill in quality standards and rubric design, building living systems with decision logs, change histories, and controls for auditability
- Experience defining quality benchmarks and validation methods for human–technology workflows, including AI-assisted processes
- Adaptability — comfort revising standards and programs as risks, products, and technology change
PREFERRED:
- Experience in language assessment, test security, remote proctoring, trust and safety, or fraud prevention in high-integrity environments
- Experience maintaining quality across large, distributed examiner/rater/reviewer/contractor populations
- Familiarity with assessment concepts such as inter-rater agreement, reliability, standardization, and performance monitoring
- Experience designing quality systems that span human judgment and technology-assisted workflows
- Familiarity with learning-management systems, assessment platforms, quality-audit tools, or operational analytics
