SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Salary: USD 194,000 - 267,000 / annual
Okta is building Digital Workers, a new suite of autonomous agents for identity operations and security. This Staff Software Engineer role reports to a team working directly with Okta's co-founder and CEO, Todd McKinnon, with the opportunity to bring this product to Okta's 20,000+ customers.
You will build the harnesses and evaluations that make these AI workers reliable. The agents investigate risk, use tools, wait for approvals, and complete tasks across enterprise systems. Your work will focus on agent execution, model quality, and turning these capabilities into software customers can trust.
Key responsibilities include:
- Build the agent harness for planning, tool use, memory, context compaction, interruption and recovery during long-running work
- Integrate OpenAI and Anthropic model APIs, selecting models based on measured quality, latency and cost with clear failure handling
- Evaluate existing harnesses and build missing pieces while keeping the harness connected to AWS workflow execution and approval state
- Keep authorization and sensitive writes behind application policy checks, requiring evidence of resulting state before marking work complete
- Build evaluation datasets and replay tests for valid findings, access decisions, app setup and AI agent onboarding
- Measure false positives, missed risks, unsafe actions, completion rate and cost; turn production failures into regression tests
- Test prompt injection, stale evidence, conflicting instructions and unsafe tool use; protect tenant boundaries in retrieval and memory
- Improve prompts, retrieval, tools and model routing; evaluate fine-tuning or post-training methods when justified
The tech stack includes Python FastAPI, AWS Step Functions, Amazon Bedrock AgentCore, PostgreSQL on Amazon RDS, pgvector, and OpenAI/Anthropic model APIs. Plans include adding a dedicated agent harness and analytics layer using ClickHouse or BigQuery.
As a Staff-level engineer, you will lead a domain such as harnesses, evaluations or model quality across workers and implement its critical components.
REQUIREMENTS:
- Strong Python engineering skills and experience shipping an LLM application that uses tools
- Hands-on experience building agent workflows or evaluation systems for tool-using LLM applications (Claude Agent SDK, OpenAI Codex SDK, Strands Agents, LangChain Deep Agents, LangGraph, or custom implementations)
- Experience integrating model APIs, structured outputs and asynchronous services; OpenAI or Anthropic experience especially relevant
- Depth in agent execution or evaluation systems, with working code and failure analysis you can explain
- Ability to distinguish a plausible model response from a supported finding or verified action
Extra credit for: open-source contributions, agent observability, security evaluations, synthetic evaluation data, fine-tuning, preference data, post-training evaluation, model distillation, identity and security workflows experience.
