SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Handshake AI works with frontier labs on their most consequential data, evaluation, and post-training challenges. The company builds systems that turn expert human knowledge into data and evaluations that improve frontier models.
Handshake Labs is building external AI products, research platforms, and customer-facing AI systems, evolving custom work into reusable products and platforms. The work spans the full post-training loop: designing evaluations and training environments, building high-quality data and feedback systems, running experiments, and turning what works into durable infrastructure.
As a Member of Technical Staff, Post-Training, you will help define and build this new organization in a broad, high-ownership role. You may come from research science, research engineering, machine learning engineering, or a closely related background; what matters is the ability to reason deeply about model improvement and turn that reasoning into reliable systems.
You will partner with researchers, domain experts, and customers to turn ambiguous post-training questions into experiments, evaluation frameworks, data pipelines, and products. Early members of the team will have unusual influence over technical direction, operating culture, and the reusable systems built.
Key responsibilities include: designing post-training systems and methodologies for frontier models (supervised fine-tuning, reinforcement learning, preference optimization, reward modeling); translating open-ended research or partner needs into clear hypotheses, experiments, and production-quality implementations; building and improving evaluation frameworks, benchmarks, training environments, data-processing pipelines, and quality-control systems; running fast, rigorous iteration loops; partnering directly with AI researchers and domain experts to develop high-signal data and feedback methods; identifying repeatable patterns and productizing them into reusable software; raising the technical bar through design judgment, communication, code quality, and mentorship; and contributing to the field through benchmarks, open-source tools, research, and technical writing.
You will work alongside engineers, researchers, operators, and builders from organizations including Scale AI, Meta, Google, Amazon, xAI, Notion, and Palantir.
REQUIREMENTS:
- 3+ years of demonstrated strength in post-training, fine-tuning, or model-evaluation work (RL, SFT, LoRA/PEFT, full fine-tuning, RLHF, DPO, PPO, reward modeling, or training environments)
- Strong Python skills and ability to write clean, efficient, scalable software
- Hands-on experience with modern ML tooling, particularly PyTorch and large-scale data, training, or evaluation workflows
- Sound experimental judgment: forming hypotheses, choosing meaningful metrics, diagnosing failures, distinguishing signal from noise
- Experience designing systems (not only implementing specifications), including ability to make tradeoffs around quality, scale, reliability, and reuse
- Comfort operating in ambiguous, fast-moving environment with substantial ownership
- Collaborative, low-ego communication and ability to work effectively with researchers, engineers, domain experts, and customers
ESPECIALLY COMPELLING:
- Building or operating large-scale ML training, inference, data, or evaluation systems
- Developing LLM/agent benchmarks, evaluation methodologies, annotation systems, or data-quality frameworks
- Research or applied work on reinforcement learning, alignment, model behavior, synthetic data, or human-in-the-loop systems
- Published research, meaningful open-source contributions, or evidence of technical leadership in ML systems or AI research
- Experience productizing research or repeated customer work into robust, reusable platforms
