SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Salary: USD 150,000 - 350,000 / annual
Prime Intellect is building the open superintelligence stack—infrastructure for frontier AI labs to train and deploy models at scale. The company has raised $150M from Founders Fund, Radical Ventures, NVIDIA, and notable operators including Andrej Karpathy and leaders from Perplexity, Harvey, Cognition, OpenAI, and others.
As a Research Engineer - Data and Evaluation, you will own the design and implementation of evaluations and environments that train Prime Intellect's models through reinforcement learning. You'll work directly with researchers to investigate model failures, from task implementation to success scoring, and build the data pipelines that generate challenging coding and agentic tasks.
Key responsibilities include:
- Design and build challenging evaluations for coding and long-horizon agents
- Build executable environments with realistic task setup and clear success criteria
- Develop and test scoring logic and verifiers, investigating reward hacks and misleading measures of progress
- Trace failures through model outputs and harness code to distinguish model limitations from environment or scoring problems
- Build and scale RL data pipelines that generate and curate challenging tasks
- Collaborate with RL researchers to test whether changes to tasks and training data improve performance on held-out tasks
You'll have significant technical ownership, working at the intersection of evaluation methodology, environment design, and model training. The role requires deep collaboration with researchers and engineers to turn ambiguous quality questions into testable measurements.
Benefits include flexible work arrangements (remote or in-person in San Francisco), visa sponsorship and relocation assistance, quarterly team off-sites and hackathons, and learning opportunities.
REQUIREMENTS:
- Strong software engineering skills in Python and experience building data pipelines, evaluation systems, or ML workflows used by others
- Hands-on experience working with language models or agents, with understanding of how training data and evaluation choices affect model behavior
- Ability to inspect complex data and model outputs, identify failure modes, and turn ambiguous quality questions into testable measurements
- Record of owning technical projects through experimentation, implementation, and validation while collaborating closely with researchers and engineers
- Hands-on experience building technical evaluations or agent environments, with engineering depth to inspect model traces and debug harness and scoring code
Relevant experience may include: large-scale synthetic data generation, dataset curation, or distributed processing for model training; evaluation methodology, executable coding tasks, agent environments, verifiers, or reward design; experiment analysis, statistical testing, contamination detection, and reproducible research tooling.
Nice to have: experience with RL or post-training data including task difficulty selection and analysis of rollout or reward distributions; contributions to Verifiers, Prime-RL, or comparable frameworks; work on model-based grading, long-horizon coding agents, or diagnosing reward exploitation.
