SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Perplexity AI is seeking a Member of Technical Staff (Software Engineer) to join the Enterprise Agent Evals & Efficiency team. This is a hands-on engineering role with end-to-end ownership focused on making Perplexity's agents better at completing real work across enterprise connectors, tools, and data.
You will own the improvement loop: define what successful work looks like, build evaluations that measure it, understand where agents fail, and ship changes that improve outcomes. The role spans building realistic enterprise evaluation environments to shipping improvements across the agent harness, tools, and skills.
Key responsibilities include:
- Building realistic enterprise evaluation environments spanning connected applications, representative datasets, user roles, and permissions, with complex multi-step workflows and reproducible system state.
- Owning evaluation infrastructure, task suites, and automated graders that measure how well agents complete enterprise work, calibrated against human judgment and grounded in end-to-end outcomes.
- Analyzing agent trajectories and production failures, identifying root causes, and shipping improvements across the agent harness, tools, and skills that measurably improve task success and reliability.
- Developing reusable skills and improving tool descriptions, schemas, and result handling so agents can select the right tools, execute workflows correctly, and recover from errors.
- Building and evaluating model-selection and routing strategies that match each task to the right model and effort level, balancing quality, latency, reliability, and cost.
- Improving agent efficiency through context management, skill loading, caching, and execution strategies that reduce wasted tokens and cost per successful task.
- Building regression gates and launch-readiness signals for model, connector, tool, skill, and harness changes, and validating that evaluation gains translate into better production outcomes.
You will work closely with AI Research, Connectors, and Platform teams to turn experimental results into measurable gains in task success, latency, and cost per successful task.
REQUIREMENTS:
- 5+ years of software engineering or machine learning engineering experience, with a track record of owning complex production systems from design through deployment and operation.
- Strong system design skills, including building reliable, scalable infrastructure for experimentation, evaluation, or data-intensive applications.
- Strong proficiency in a backend language such as Python, Go, or Rust, and the ability to work effectively across a multi-language production stack.
- Hands-on experience building, evaluating, or improving LLM-based applications or agents, including tool use and multi-step workflows.
- Deep expertise in at least one of the following: evaluation infrastructure and benchmark design; agent harnesses and tool interfaces; realistic test environments spanning multiple systems; or model selection and inference efficiency.
- Strong experimental and statistical judgment, including choosing representative tasks, defining meaningful success criteria, diagnosing failures, and distinguishing real improvements from noise.
- Sound technical judgment in balancing task quality, latency, reliability, and cost, with a track record of translating evaluation findings into shipped improvements.
- Track record of leading ambiguous, cross-team initiatives from architecture through production adoption, setting technical direction, and raising engineering quality through mentorship.
About Perplexity AI
AI / Data / Infrastructure — AI answer engine and search product for consumers and enterprises.
