SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Prolific is building human data infrastructure for AI development. As a Senior Product Engineer on the Agentic Training & Evals team, you'll be one of the early engineers building the infrastructure to evaluate AI agents at scale, capturing agent traces alongside real human expert work in secure environments.
This is a genuine 0→1 role with significant influence over technical direction and product strategy. You'll work closely with engineering, product, and AI research teams to turn ambiguous problems into scalable systems. Your work will have direct, visible impact on how AI agents are trained, benchmarked, and improved.
Key responsibilities include:
- Building tooling for capturing and processing data from agents and humans performing real-world tasks at significant scale
- Solving hard problems around compute, orchestration, scaling, security, and reliability
- Developing approaches for training, benchmarking, and evaluating AI agents
- Working with emerging agentic AI technologies and rapidly evolving models
- Making pragmatic technical decisions in highly ambiguous environments
- Partnering with product engineering and AI research to turn customer problems into working products
- Driving direct impact in a fast-moving, high-ownership, cross-functional team
Key technologies include Google Cloud Platform, Python, JavaScript, TypeScript, FastAPI, Vue.js, container-based and serverless architectures, PostgreSQL, GitHub Actions, Kubernetes, and DataDog.
Requirements:
- Strong experience building scalable infrastructure or distributed systems in the cloud
- Solid understanding of how systems are deployed, scaled, and secured
- Hands-on experience integrating and evaluating AI models or agents
- Interest in how AI agents are trained, evaluated, and improved
- Strong product mindset with focus on delivering business and customer value
- Comfort with ambiguity and ability to move quickly from idea to working system
- Strong communication and collaboration skills
- Python experience useful but not essential
- Bonus: experience with image, video, or audio processing
