SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Apna is seeking an experienced QA Lead with 7+ years of software quality assurance expertise to own the complete quality lifecycle for product-based technology initiatives. This is a hands-on leadership role requiring deep expertise in manual testing, automation testing, Python, and emerging AI/ML testing methodologies.
Key Responsibilities:
- Own overall quality strategy for assigned products and engineering teams, leading manual and automation testing across web, mobile, APIs, backend services, and AI features.
- Design, develop, and maintain scalable automation frameworks using Python with tools like Pytest, Selenium, Playwright, Appium, and Robot Framework.
- Create comprehensive test plans, scenarios, cases, and release-quality reports; perform functional, regression, integration, API, database, exploratory, and performance testing.
- Define and execute testing strategies for AI/ML and Generative AI features including chatbots, recommendation systems, semantic search, summarization, classification, and content generation.
- Validate AI-generated responses for accuracy, relevance, consistency, completeness, safety, and business-rule compliance; test for hallucinations, prompt injection, data leakage, bias, and edge cases.
- Build automated evaluation frameworks and datasets for LLM and AI-powered features; test RAG workflows including document retrieval, context relevance, response grounding, and citation accuracy.
- Validate AI model and third-party LLM API integrations for reliability, latency, error handling, rate limits, token usage, and cost.
- Establish baseline quality metrics and regression suites for AI outputs; define and track defect leakage, automation coverage, regression effectiveness, release readiness, AI response accuracy, hallucination rate, and latency.
- Review product requirements, prompts, workflows, and technical designs to identify gaps and risks early; work closely with Product, Developers, DevOps, Data Scientists, and AI/ML Engineers.
- Lead release validation, QA sign-off, production sanity testing, and post-release monitoring; analyze production defects and implement preventive measures.
- Mentor QA engineers and promote quality-first culture across teams.
Required Qualifications:
- 6+ years in software testing and QA with strong hands-on expertise in manual and automation testing.
- Proficiency in Python for automation frameworks; experience with Pytest, Selenium, Playwright, Appium, Robot Framework.
- API testing experience (Postman, Python Requests, REST Assured); database testing and SQL proficiency.
- Strong understanding of testing methodologies, QA processes, SDLC, STLC, and functional/integration/regression/system/exploratory/end-to-end testing.
- CI/CD pipeline integration experience; hands-on with Git, Jenkins, GitHub Actions, Jira.
- Product-based company experience testing customer-facing products at scale.
- Understanding of AI/ML concepts and experience testing AI-powered or Generative AI features; knowledge of LLM behavior, hallucinations, context limitations, and prompt sensitivity.
- Ability to design test datasets, evaluation criteria, and quality metrics for AI outputs.
- Strong analytical, debugging, problem-solving, and risk-identification skills; excellent communication and team-leadership capabilities.
Desirable:
- Experience with LLM-based applications, AI chatbots, RAG systems, recommendation engines, semantic search.
- AI evaluation tools (LangSmith, DeepEval, Ragas, Promptfoo, TruLens).
- Familiarity with OpenAI, Gemini, Claude, or open-source LLM platforms.
- Prompt engineering and automated prompt-regression testing knowledge.
- Responsible AI, privacy, security, fairness, and bias evaluation experience.
- Performance testing tools (JMeter, Locust, k6); microservices and distributed systems testing.
- Cloud platforms (GCP, AWS, Azure); Docker, Kubernetes, Kafka; monitoring tools (Grafana, Kibana, Datadog).
- Recruitment technology, marketplaces, SaaS, or high-scale product experience.
