SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Karat is the world's largest technical interviewing platform, serving companies like PayPal, Atlassian, and Citi. The Interview Content team designs, writes, tests, and monitors technical interview questions and scoring rubrics used across the platform.
As Senior Content Analytics Manager, you will own the health and performance of Karat's technical interview content. You'll define metrics, establish quality standards, and drive continuous improvement of signals that ensure fair, reliable, and consistent interview outcomes. You'll leverage AI solutions to analyze, automate, and iterate on interview content while proactively identifying issues, investigating root causes, and owning corrective actions.
Key responsibilities include:
- Design, build, and maintain dashboards and automated monitoring systems for content health and performance visibility
- Own measurement and monitoring of content quality, including score distributions, content equivalence, cheating indicators, and score clustering
- Leverage AI solutions and agentic automation to scale content quality monitoring and improvement efforts
- Independently develop theories, perform root-cause analyses, and validate solutions using data-driven processes
- Own feedback loops with Content Development and Architect teams to prevent recurring issues
- Monitor key business and content-performance indicators, proactively escalating concerning trends to leadership
- Serve as subject matter expert on content quality, communicating trends, risks, and recommendations to leadership and executives
You bring 6+ years of analytics or strategy experience with demonstrated ownership of business outcomes. You're proficient with modern AI tools (Claude, GPT, etc.) for complex data analysis and AI-agent automation workflows. You have deep expertise in SQL and large-scale data analysis (Redshift experience preferred). You're fluent in data visualization and BI tools like Metabase, with strong ability to communicate insights to diverse stakeholders. You demonstrate exceptional critical thinking, problem-solving, and executive-level communication skills. You're comfortable in ambiguous environments with significant autonomy. Native-level English proficiency required. Python experience is a plus.
