SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Salary: USD 142,800 - 193,200 / annual
Duolingo is seeking a Proctor Quality and Learning Manager to own the end-to-end quality standards and learning systems for the Duolingo English Test (DET) global proctor network. This role sits within DET Proctoring Operations and carries functional leadership over Training Specialists and adjudicators.
You will define what "correct" looks like in proctor decision-making, operating recurring Proctor Certification Test cycles and resolving disputed decisions to ensure consistency and auditability. You'll build and run an ongoing learning program with a forward-looking training calendar, monthly interventions, and targeted training in response to quality signals, security threats, or workflow changes. Using calibration results, audits, and performance trends, you'll measure training effectiveness and identify when standards need clarification, retraining, or guidance updates.
You'll guide Training Specialists and adjudicators by setting expectations, assigning deliverables, reviewing outputs, and operating onboarding and certification programs that ramp new specialists quickly. You'll maintain durable sources of truth—keeping calibration, adjudication, and training runbooks current and auditable. As security threats, technology, and workflows evolve, you'll translate these changes into clear standards, training, and quality controls in partnership with Psychometrics, Security, Operations, Product, and Engineering teams.
You'll represent the proctoring craft by bringing frontline proctor evidence into product experiments and workflow decisions, assessing whether launched changes preserve decision quality. You'll also connect human and technology-assisted quality by establishing benchmarks for AI-assisted workflows and validating automated capabilities against real sessions before deployment.
REQUIREMENTS:
- 5+ years in assessment development, scoring/rater quality, certification, quality assurance, or a field involving complex, consequential judgments; experience aligning reviewers to a shared standard through calibration programs
- Adult-learning program experience spanning onboarding, certification, ongoing training, remediation, and effectiveness measurement (beyond one-off materials)
- Exceptional attention to detail and strong judgment in ambiguous cases; ability to produce clear, defensible decisions and guidance
- Analytical fluency to interpret quality data, disagreement patterns, and performance trends and translate them into practical interventions
- Clear written and verbal communication; ability to turn complex standards into precise, usable guidance and explain difficult calls
- Cross-functional leadership without formal authority; experience directing contractors or partners toward clear accountability and outcomes
- Demonstrated skill in quality standards and rubric design, building living systems with decision logs, change histories, and controls for auditability
- Experience defining quality benchmarks and validation methods for human–technology workflows, including AI-assisted processes
- Adaptability—comfort revising standards and programs as risks, products, and technology change
PREFERRED:
- Experience in language assessment, test security, remote proctoring, trust and safety, or fraud prevention in high-integrity environments
- Experience maintaining quality across large, distributed examiner/rater/reviewer/contractor populations
- Familiarity with assessment concepts such as inter-rater agreement, reliability, standardization, and performance monitoring
- Experience designing quality systems that span human judgment and technology-assisted workflows
- Familiarity with learning-management systems, assessment platforms, quality-audit tools, or operational analytics
