SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Salary: USD 165,000 - 226,000 / annual
Okta is building Digital Workers, a suite of autonomous AI agents for identity operations and security. You will join the team that created Jeeva AI, working directly with co-founder and CEO Todd McKinnon to bring this product to Okta's 20,000+ customers.
In this role, you will own an agent capability or evaluation pipeline with measurable improvements. You'll build the harness and evaluation systems that make autonomous agents reliable for enterprise use. The work spans agent execution (planning, tool use, memory, context compaction, interruption and recovery), model integration (OpenAI and Anthropic APIs), and rigorous quality measurement.
Key responsibilities include:
- Build the agent harness for long-running autonomous work, including planning, tool use, memory management, and recovery mechanisms
- Integrate and select between OpenAI and Anthropic models based on measured quality, latency, and cost
- Evaluate existing harnesses and implement missing pieces while maintaining integration with AWS Step Functions and approval workflows
- Enforce authorization and sensitive-write policies; require evidence of state before marking work complete
- Build evaluation datasets and replay tests for valid findings, access decisions, app setup, and agent onboarding
- Measure false positives, missed risks, unsafe actions, completion rates, and costs; convert production failures into regression tests
- Test for prompt injection, stale evidence, conflicting instructions, and unsafe tool use; protect tenant boundaries
- Improve prompts, retrieval, tools, and model routing; evaluate fine-tuning when justified
You will operate with startup speed and ownership while leveraging the resources of a public company. The core stack is Python FastAPI, AWS Step Functions, Amazon Bedrock AgentCore, PostgreSQL on Amazon RDS, and pgvector. Plans include adding a dedicated agent harness and analytics layer using ClickHouse or BigQuery.
Requirements:
- Strong Python engineering skills and proven experience shipping an LLM application that uses tools
- Hands-on experience building agent workflows or evaluation systems for tool-using LLM applications (Claude Agent SDK, OpenAI Codex SDK, Strands Agents, LangChain Deep Agents, LangGraph, or custom implementations)
- Experience integrating model APIs, structured outputs, and asynchronous services; OpenAI or Anthropic experience especially relevant
- Depth in agent execution or evaluation systems, with working code and failure analysis you can explain
- Ability to distinguish a plausible model response from a supported finding or verified action
Extra credit for: open-source contributions, agent observability, security evaluations, synthetic evaluation data, fine-tuning, preference data, post-training evaluation, model distillation, identity and security workflows, or foundation-model research experience.
