SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
ASAPP is building AI-powered customer experience solutions and seeking a Senior Product Manager to own the observability and quality assurance layer of its platform. This is a non-reporting individual contributor role focused on making ASAPP's GenerativeAgent (GA) behavior transparent, trustworthy, and actionable for customers.
You will own four interconnected product areas:
1. **Scenario Creation via Simulation Agent**: Lead the tools that enable builders to generate and curate realistic test scenarios, allowing configuration changes to be validated against representative conversations before production deployment.
2. **Evaluation Surfaces**: Define how the platform detects, surfaces, and categorizes GA mistakes at scale, and how structured data extraction is evaluated for accuracy—converting noisy conversational output into clear, actionable quality signals for builders and customers.
3. **Previewer Tool**: Own the interface builders use to see exactly how their configurations appear in simulated conversations, closing the feedback loop between configuration changes and understanding downstream impact.
4. **Conversation Explorer**: Lead the primary surface for reviewers to examine full production conversations—including agent reasoning and actions—balancing investigative depth with high-volume auditing speed.
Additionally, you will build the business case for simulation-driven quality by partnering with Sales, Customer Success, and Research to demonstrate how pre-production testing and post-production review reduce mistakes, accelerate iteration, and strengthen GA performance defensibility. You'll work closely with Research and Engineering on underlying agent architecture—reasoning traces, tool calls, scenario generation methods—ensuring product decisions are grounded in technical reality. You'll also manage the trust layer of the review experience by proactively identifying and resolving issues that erode confidence in what builders and reviewers see, such as desyncs between simulated and real behavior, incomplete traces, and unclear mistake categorization.
The role requires extreme ownership, high-velocity outcome focus, effective cross-functional collaboration, and continuous feedback loops. You'll influence without authority across engineering, research, design, sales, and customer success.
**Requirements:**
- 5+ years of product management experience with meaningful time on testing, quality, or review/analytics products in enterprise B2B SaaS
- Direct experience building products around LLMs, AI agents, or applied AI systems—able to reason about agentic behavior, prompt and scenario design, or model output evaluation to make sharp product tradeoffs
- Experience shipping tools that simulate, test, or preview system behavior before production, with judgment about when simulated behavior reliably proxies real behavior
- Experience building review or audit tooling for complex, unstructured content and translating ambiguous quality signals into clear, defensible categorizations
- Comfort partnering with research and engineering teams on system internals (reasoning traces, tool-use logs, evaluation methodology) to ground product decisions in technical reality
- Track record of running discovery with internal or external users and translating findings into structured requirements (evals, quality metrics, review workflows)
- Excellent written and verbal communication; ability to write clear PRDs, release announcements, and present directly to customers
**Nice-to-Have:**
- Experience in contact centers, conversational AI, or customer experience (CX) analytics
- Experience with evaluation frameworks or methodologies for generative AI systems (offline evals, human-in-the-loop review, automated mistake detection)
- Prior experience in a startup or fast-scaling environment where role scope evolved with company growth
