SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Dialpad is an AI platform for customer experience that enables AI agents to resolve customer problems in real time across voice and digital channels. The company's AI agents learn from human agents and improve with every interaction, helping organizations deliver better experiences and increase operational efficiency.
As a Senior Software Development Engineer in Test (SDET) in Agentic QA, you will own the test automation and quality frameworks supporting Dialpad's AI Voice Agent services. You will develop comprehensive automated tests for end-to-end product experiences spanning frontend UI, backend services, APIs, and audio/text interactions. Your responsibilities include testing orchestration flows, agent configuration experiences, and guardian safeguards to ensure robust automated coverage for functionality, performance, reliability, and user experience.
Key responsibilities include:
- Owning end-to-end quality for agentic features and workflows, including strategy, development, execution, and release qualification
- Designing and building automation tooling and frameworks specifically for AI/LLM-driven systems, including prompt flows, agent orchestration, and tool integrations
- Developing and maintaining evaluation frameworks (evals) to measure response quality, accuracy, and hallucination rates
- Driving automation coverage (80%+ for critical AI workflows) using both deterministic and probabilistic validation approaches
- Integrating AI quality checks into CI/CD pipelines with fast feedback cycles (under 15 minutes for PR validation)
- Building tooling for LLM observability and debugging, including prompt tracing and response analysis
- Partnering with Applied AI teams on prompt engineering, model selection, and evaluation strategies
You will develop substantial amounts of automated test infrastructure and partner deeply with the development team to make the fast-growing AI platform more testable, more stable, and more delightful for customers. This position is based at one of Dialpad's Canadian offices and reports to a QA Engineering Manager in the United States.
