SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Mimica Automation is seeking a Senior Quality Engineer to establish quality foundations for a newly formed, multidisciplinary chatbot team building conversational AI components. As the team's founding Quality Engineer, you will define quality practices from the outset for a product characterized by dynamic, non-deterministic LLM-powered outputs.
Your responsibilities include developing and evolving quality practices, test strategy, and release processes for the Chatbot product. You will establish and run an output-evaluation framework for assessing chatbot quality, reliability, and regressions. A core challenge is testing dynamic and non-deterministic LLM-based systems, designing pragmatic approaches for validating behavior beyond conventional deterministic test cases. You will identify, test, and improve guardrails for AI-specific risks, including prompt-injection scenarios and unsafe or unexpected model behavior.
You will partner with software and machine learning engineers through code reviews, technical discussions, and collaborative debugging to improve quality earlier in the development lifecycle. You'll investigate testing gaps, research relevant methods and tools, and make evidence-based recommendations. You will design proof-of-concepts for new testing approaches or tooling, then help productionize solutions that create the most value. Additional responsibilities include supporting the evolution of release testing with clearer quality signals and release-readiness criteria, contributing performance-testing thinking for LLM-powered workflows, and sharing knowledge to build the Quality Chapter across the wider engineering organization.
Required qualifications include strong hands-on experience in software quality engineering, SDET, test automation, or closely related engineering roles. You need experience testing chatbot, conversational AI, RAG, LLM-powered, or other systems with dynamic and non-deterministic behavior. Proficiency in at least one programming language with technical depth to understand production code, contribute to code reviews, and collaborate effectively with software engineers is essential. An experimental, investigative mindset is critical—you must explore uncertain problem spaces, assess alternatives, and turn findings into practical improvements. Strong written and verbal communication skills and pragmatic collaboration with Product, Engineering, and ML colleagues are required.
Bonus qualifications include experience with AI security and safety testing, conversational AI or chatbot platforms, being the first Quality Engineer in a team, mentoring engineers, and staying current with industry developments.
