SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
OpenAI's Implicit Signals team builds classification and evaluation systems that provide fast, reliable feedback to post-training and research teams on model behavior. The team combines machine learning and production engineering to create accurate, efficient, and scalable capabilities that help researchers assess model changes and improve products.
In this role, you will develop new classification and evaluation capabilities and bring them into production. You'll translate open-ended questions about model behavior into well-defined ML problems, build methods to address them, and establish how reliably those methods work. This is a full-lifecycle role spanning supervised fine-tuning, distillation, evaluation, deployment, and production inference.
Key responsibilities include:
- Building, deploying, and operating production classification and evaluation systems that provide feedback to post-training and research teams
- Optimizing production inference across model serving, batching, caching, throughput, latency, compute utilization, and cost
- Training and optimizing classifiers using supervised fine-tuning and distillation, with rigorous quality and efficiency validation
- Owning reliability and observability of deployed classifiers, diagnosing production issues, and iteratively improving performance
- Designing rigorous evaluations assessing accuracy, calibration, generalization, and measurement bias while guiding production tradeoffs
- Partnering with software engineers to integrate models into reliable execution systems and support safe rollouts
- Building reusable engineering tools that accelerate classifier training, evaluation, deployment, and iteration
- Working with post-training researchers and data scientists to translate needs into modeling objectives, evaluation criteria, and production requirements
You'll work closely with both research and engineering teams on open-ended problems in a fast-paced environment, using AI agents to move quickly across multiple workstreams.
REQUIREMENTS:
- Experience taking ML projects from problem formulation and experimentation through production deployment
- Strong foundations in supervised learning, statistics, and experimental design
- Practical experience evaluating models, investigating systematic errors, and working with imperfect labels
- Strong software engineering skills and ability to build maintainable production ML systems
- Experience reasoning about inference performance and tradeoffs between quality, latency, and compute cost
- Ability to combine careful scientific judgment with practical approach to shipping and operating software
- Comfort working closely with researchers and engineers on open-ended problems
- Fluency in using AI agents to move quickly across multiple workstreams in a fast-paced environment
- Plus: experience with model distillation, representation learning, calibration, or efficient inference
