SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Preference Model is a superintelligence data research company building reinforcement learning environments for training capable, better-aligned AI systems. The company has raised $16M in seed funding led by a16z and counts Anthropic alumni on its founding team.
You will join as a Member of Technical Staff - Research Engineer focused on post-training, in a role that blends research and engineering. Your responsibilities will include:
- Training and evaluating models on proprietary RL environments to validate data quality, identify gaps in task coverage, and close feedback loops between environment design and model capability.
- Architecting and optimizing RL training infrastructure, from training abstractions to distributed experiment management, using frameworks like Verl, OpenRLHF, or similar. You'll help scale systems to handle increasingly complex research workflows.
- Designing, implementing, and testing training environments, evaluations, and methodologies for RL agents.
- Profiling and optimizing end-to-end training runs—from data loading through reward computation—to maximize experiment throughput and shorten research iteration cycles.
The company is investigating how far self-directed learning can be pushed, particularly for models that can train themselves on tasks they are not yet good at. You'll work at the frontier of post-training on large language models, implementing novel approaches and shaping research directions.
Preference Model offers competitive cash and equity compensation (>90th percentile), ownership and autonomy in a fast-moving startup environment, opportunity to work with top machine learning engineers, comprehensive health/vision/dental benefits, 401K match, daily onsite lunch, weekly snack orders, and visa sponsorship & relocation support.
REQUIREMENTS:
- Experience running end-to-end LLM post-training pipelines on models at least 7B in size
- Proficiency in Python and PyTorch or JAX
- Experience with at least one modern RL training framework
- Experience building and operating ML infrastructure at scale
ADDITIONAL QUALIFICATIONS (nice-to-have):
- Experience evaluating model outputs and building reward or evaluation signals
- Ability to stay current on post-training research and translate papers into running code
- Strong opinions (loosely held) about structuring RL training code for reproducibility and fast iteration
- Ability to balance research exploration with engineering rigor
- Strong systems design and communication skills
Note: PhD or extensive publications not required. The company values adaptability, exceptional communication, and collaboration skills.
