SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Dialpad is an AI platform for customer experience that powers AI voice agents and human agents on a unified platform. The company serves market-leading brands including Netflix, Motorola Solutions, and Randstad, and is backed by top-tier VCs including Andreessen Horowitz and GV.
As a Senior SDET in Agentic QA, you will own test automation and quality frameworks for Dialpad's AI Voice Agent services. You'll develop comprehensive automated tests spanning frontend UI, backend services, APIs, and audio/text interactions. Your work will cover orchestration flows, agent configuration experiences, and guardian safeguards to ensure robust coverage across functionality, performance, reliability, and UX.
Key responsibilities include:
- Owning end-to-end quality for agentic features, from strategy through release qualification
- Designing and building automation tooling and frameworks specifically for AI/LLM-driven systems, including prompt flows, agent orchestration, and tool integrations
- Developing and maintaining evaluation frameworks (evals) to measure response quality, accuracy, and hallucination rates
- Driving automation coverage (80%+ for critical AI workflows) using both deterministic and probabilistic validation approaches
- Integrating AI quality checks into CI/CD pipelines with fast feedback cycles (under 15 minutes for PR validation)
- Building tooling for LLM observability and debugging, including prompt tracing and response analysis
- Partnering with Applied AI teams on prompt engineering, model selection, and evaluation strategies
You'll develop substantial test infrastructure and work closely with the development team to make the fast-growing AI platform more testable, stable, and delightful for customers. This role reports to a QA Engineering Manager in the United States and is based at one of Dialpad's Canadian offices in Kitchener.
