SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Salary: USD 121,000 - 166,000 / annual
Okta's People Operations & Digital Solutions team is seeking a Senior AI Analyst to train, evaluate, and improve AI models and AI-powered enterprise solutions. Reporting to the Senior Director of People Operations & Digital Solutions, you will serve as the subject matter expert for AI solutions across the organization, focusing on employee support chatbots, agentic AI coaches, and future AI assistants.
You will be responsible for training and continuously improving AI-powered experiences by analyzing conversational insights, optimizing AI performance, and ensuring AI assistants deliver accurate, relevant, and trusted responses. Key responsibilities include:
- Train, test, and evaluate AI-powered assistants, chatbots, and agentic AI solutions to ensure accurate and reliable employee experiences
- Design and optimize prompts, conversational flows, intents, responses, and training scenarios across enterprise support platforms
- Evaluate AI outputs for accuracy, relevance, completeness, tone, and consistency while identifying and mitigating hallucinations, response gaps, or misinformation
- Monitor and analyze AI performance metrics such as containment, resolution, user satisfaction, engagement, and escalation rates; identify trends, emerging intents, knowledge gaps, and automation opportunities from conversational data
- Apply ServiceNow knowledge to translate complex business processes into intuitive conversational scenarios, training data, and evaluation criteria; collaborate with ServiceNow administrators, developers, analysts, and business stakeholders
- Create and maintain AI training materials, test cases, evaluation criteria, and sample conversations; partner with Knowledge Management teams to improve content quality and AI readiness
- Support testing and rollout of new AI capabilities; provide insights and recommendations based on AI performance, user feedback, and analytics
- Stay current with developments in generative AI, prompt engineering, conversational AI, agentic AI, and responsible AI
This role offers the opportunity to work at the intersection of cutting-edge generative AI and enterprise technology, directly impacting how thousands of Okta employees interact with digital support.
**Requirements:**
- Experience in AI training, prompt engineering, conversational design, NLP, or AI optimization within employee or customer support environments
- Proven experience training, evaluating, and optimizing AI models, conversational agents, or LLM-powered assistants, including agentic AI solutions
- Strong understanding of prompt engineering, AI evaluation, model optimization, and responsible AI practices
- Demonstrated ability to analyze conversational data and performance metrics to drive measurable improvements
- Working knowledge of ServiceNow or a similar enterprise service-management platform
- Strong analytical skills and experience with tools such as SQL, Excel, Power BI, Tableau, or ServiceNow Analytics
- Experience with enterprise conversational AI platforms, knowledge management systems, or AI-enabled search technologies
- Excellent written and verbal communication skills with the ability to translate technical and business requirements into clear, user-focused content and influence cross-functional stakeholders
- Understanding of knowledge management, digital employee support, and omnichannel service experiences
**Preferred:**
- Strategic, proactive, and solution-oriented mindset with a passion for continuous improvement
- Ability to manage multiple priorities while working independently and influencing cross-functional stakeholders
- Curiosity for emerging AI technologies and commitment to driving innovation
