SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Preference Model is a superintelligence data research company backed by $16M in seed funding from a16z. The founding team includes former members of Anthropic's data team who built infrastructure and datasets for Claude. The company focuses on building reinforcement learning environments to train capable, better-aligned AI systems by improving the reward signals and training objectives that shape model behavior.
In this role, you will design and build RL environments and reward functions that produce clean, learnable signals for frontier models on ML research and engineering tasks. You'll teach frontier models to perform the work of ML engineers and researchers at leading labs. This is a blend of research and engineering work requiring you to stay current with cutting-edge research, develop novel approaches, and implement them in production code.
You will join the Capabilities org, a small, high-ownership team contributing directly to the data layer powering frontier LLM capability. Your responsibilities include:
- Designing and implementing RL environments and reward functions for frontier models
- Building deep expertise across the frontier of ML research, training, and inference infrastructure
- Conducting experiments and evaluations
- Delivering work into production training runs
- Collaborating with researchers and engineers to brainstorm and create new tools to improve the environment building process
- Taking full ownership and autonomy of the environments you build
The company offers competitive cash and equity compensation (>90th percentile), ownership and autonomy in a fast-moving startup environment, opportunity to work with top ML engineers, comprehensive health/vision/dental benefits, 401K match, daily onsite lunch, weekly snack orders, and visa sponsorship & relocation support.
QUALIFICATIONS (REQUIRED):
- 5+ years of experience working in machine learning or research, primarily on LLMs and transformer models
- Strong ML fundamentals and broad research interests; ability to read papers/tutorials deeply and translate them into RLVR problems
- Proficiency in Python and systems programming, plus at least one of PyTorch or JAX
- Problem-solving mindset with ability to take ownership and drive solutions end-to-end
- Passion for staying current with the rapidly evolving ML infrastructure landscape
- Ability to meet throughput expectations and respond quickly to feedback
PREFERRED QUALIFICATIONS:
- Expert knowledge in an active DL/ML research area with publications or public code; research experience (PhD, MS) is a big plus
- Deep understanding of transformer internals and training/inference of modern LLMs
- Experience with inference libraries (vLLM, SGLang, etc.)
- Strong expertise in kernel development (CUDA, Triton, Pallas)
- Experience building complex interactive RL environments
