SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Perplexity is hiring a Member of Technical Staff (Software Engineer) for the Enterprise Agent Evals & Efficiency team. This is a hands-on engineering role focused on making Perplexity's AI agents better at completing real work across enterprise systems.
You will own the full improvement loop: defining what successful agent work looks like, building evaluations to measure it, identifying failure modes, and shipping improvements. Your work spans building realistic enterprise evaluation environments with complex multi-step workflows, developing evaluation infrastructure and automated graders, analyzing agent trajectories and production failures to identify root causes, and shipping improvements across the agent harness, tools, and skills.
Key areas of impact include:
- Building realistic enterprise evaluation environments spanning connected applications, representative datasets, user roles, and permissions with reproducible system state
- Owning evaluation infrastructure, task suites, and automated graders calibrated against human judgment and grounded in end-to-end outcomes
- Analyzing agent failures and shipping improvements that measurably increase task success and reliability
- Developing reusable skills and improving tool descriptions, schemas, and result handling so agents select and execute correctly
- Building model-selection and routing strategies that balance quality, latency, reliability, and cost
- Improving agent efficiency through context management, caching, and execution strategies that reduce token waste and cost
- Building regression gates and launch-readiness signals for model, connector, tool, and harness changes
You'll work cross-functionally with AI Research, Connectors, and Platform teams to turn experimental results into measurable gains in task success, latency, and cost per successful task.
Qualifications:
- 5+ years of software engineering or machine learning engineering experience with a track record of owning complex production systems from design through deployment and operation
- Strong system design skills, including building reliable, scalable infrastructure for experimentation, evaluation, or data-intensive applications
- Strong proficiency in a backend language such as Python, Go, or Rust, and ability to work effectively across a multi-language production stack
- Hands-on experience building, evaluating, or improving LLM-based applications or agents, including tool use and multi-step workflows
- Deep expertise in at least one of: evaluation infrastructure and benchmark design; agent harnesses and tool interfaces; realistic test environments spanning multiple systems and stateful workflows; model selection, routing, and inference efficiency
- Strong experimental and statistical judgment, including choosing representative tasks, defining success criteria, diagnosing failures, and distinguishing real improvements from noise
- Sound technical judgment balancing task quality, latency, reliability, and cost, with a track record of translating findings into shipped improvements
- Track record of leading ambiguous, cross-team initiatives from architecture through production adoption, setting technical direction, and raising engineering quality through mentorship
