SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Backcountry is building an agentic AI experimentation platform to move from manual, low-volume A/B testing to an always-learning system that designs, launches, monitors, and optimizes experiments across their ecommerce portfolio. As Senior Product Manager, you will own this platform from inception—setting the roadmap, defining where AI agents act autonomously versus where humans stay in the loop, and driving measurable gains in conversion and revenue.
You will own the vision and roadmap for the agentic experimentation platform, deciding what gets built first, what waits, and what to buy versus build. You'll define how AI agents participate across the full experiment lifecycle: hypothesis generation, variant creation, test setup and QA, live monitoring, and ship-or-rollback decisions. You'll set guardrails that keep autonomous testing safe through statistical standards, sample ratio mismatch and anomaly checks, revenue and margin thresholds, approval workflows, and automatic rollback mechanisms.
You'll build adaptive optimization into the platform using multi-armed and contextual bandits so traffic shifts toward winning experiences while tests are still running. You'll partner with engineering and data science on core infrastructure including assignment and bucketing, feature flags, event tracking, the metrics layer, and the stats engine built on GA4 and BigQuery. From day one, you'll define and instrument platform success metrics—test velocity, win rate, revenue impact, and result trustworthiness.
You'll make experimentation self-serve for merchandising, marketing, UX, and engineering teams so they can launch and read out tests without a specialist. You'll create standards and playbooks for metric definitions, test design, and readouts so results are consistent, comparable, and trusted across every site. You'll partner across engineering, design, analytics, merchandising, and marketing to turn business goals into a shipped platform and a culture of testing.
At six to twelve months, success looks like a working platform that runs trusted experiments across multiple sites, a significant increase in testing velocity without sacrificing statistical rigor, and revenue and conversion improvements you can trace directly to platform-run experiments. This is a lean team where you will own a lot, move fast, and make decisions with full end-to-end responsibility.
REQUIREMENTS:
- 5+ years of product management experience, including ownership of a platform, data, or internal tooling product
- Hands-on experience running or building A/B testing and experimentation programs at scale in ecommerce or other high-traffic digital products
- Strong statistical grounding in experiment design: power and sample size calculation, significance, peeking, multiple comparisons, and guardrail metrics
- Hands-on experience building with AI and LLMs, including agentic workflows, prompt design, tool integrations, and evaluating AI output quality
- Working knowledge of experimentation infrastructure—feature flagging, randomization, event tracking, and warehouse-native metrics
- Strong analytical skills with hands-on use of SQL, GA4, BigQuery, Looker, or similar tools
- Demonstrated impact on conversion, revenue, or testing speed with measurable results
- Excellent communication skills and the ability to influence engineering, data, and business stakeholders
PREFERRED:
- Experience with experimentation platforms such as Optimizely, Statsig, Eppo, GrowthBook, LaunchDarkly, or an in-house equivalent
- Experience with adaptive experimentation—multi-armed or contextual bandits, Bayesian methods, or variance reduction techniques like CUPED
- Experience testing AI or ML-driven experiences such as search, recommendations, or personalization
- Experience running experimentation across multiple brands or storefronts on a shared platform
- Background in outdoor, specialty, or multi-brand ecommerce
