SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Extreme Networks, a global networking leader with 50,000+ customers, is seeking a Senior AI Behavioral Scientist & Product Analyst to evaluate and optimize AI agents and autonomous systems within Agent ONE, their AI operating model embedded in Extreme Platform ONE.
You will own the instrumentation, analytics, and reporting that demonstrate Agent ONE's adoption, value, and impact on customer retention and revenue. Sitting at the intersection of Product, Engineering, and UX Research, you'll define what to measure, ensure proper instrumentation, build dashboards, and translate data insights into compelling narratives about product success and customer value.
Key Responsibilities:
• Design and conduct comprehensive behavioral studies evaluating AI agent performance across usability, task completion, error recovery, and user satisfaction
• Apply HCI and behavioral science frameworks to understand user interactions with AI systems and identify friction points
• Assess agent reliability, decision transparency, and failure modes from user and systems perspectives
• Develop user analytics frameworks tracking AI agent effectiveness, including KPIs for success, user trust, task completion rates, and cognitive load
• Design and implement instrumentation to collect behavioral and interaction data from user sessions
• Conduct statistical analysis, A/B testing, and cohort analysis to measure AI agent impact and validate product improvements
• Evaluate AI agents holistically across performance dimensions: accuracy, latency, cost efficiency, scalability, and robustness
• Model and communicate trade-offs between performance dimensions (e.g., accuracy vs. speed, cost vs. quality)
• Identify bottlenecks and systemic issues affecting user experience across the full AI-user interaction lifecycle
• Translate behavioral findings and analytics into actionable product recommendations and roadmap priorities
• Partner with product managers, engineers, and designers to validate hypotheses and guide feature development
• Create dashboards, reports, and narratives communicating behavioral insights to stakeholders and leadership
Required Qualifications:
• Prior experience evaluating AI systems, machine learning models, chatbots, or autonomous agents
• 4+ years of professional experience in user analytics, product analytics, behavioral research, or HCI research roles
• Strong foundation in human-computer interaction (HCI) principles and behavioral science fundamentals (cognitive load, mental models, task completion, user trust)
• Demonstrated proficiency with quantitative analytics: SQL, Python/R for statistical analysis, and familiarity with A/B testing and experimental design
• Experience designing and instrumenting analytics tracking, defining success metrics, and interpreting complex datasets
• Ability to think systemically about product performance and understand interactions between multiple variables affecting user outcomes
• Excellent communication skills with ability to translate technical findings into clear, compelling narratives for non-technical stakeholders
Preferred Qualifications:
• Background in formal usability testing, user research, or user testing methodologies
• Experience with analytics platforms (Amplitude, Mixpanel, Looker, Tableau) or BI tools
• Familiarity with systems thinking frameworks, causal inference, or econometric modeling
• Experience collaborating with ML/AI engineers or understanding of AI/ML capabilities and limitations
• Graduate degree in psychology, HCI, behavioral science, statistics, or related field
• Experience measuring LLM or conversational-AI systems (evaluation signals such as latency, refusal/quality scoring, conversation-trajectory classification)
• Familiarity with networking, IT operations, or B2B SaaS, and operational KPIs like MTTR, uptime, or escalation rate
• Experience with modern data stack (Snowflake, dbt) and light scripting in Python or R
• Exposure to NPS, SUS, or other structured voice-of-customer programs; comfort with sentiment/theme analysis of free-text feedback
