SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Reflection AI is a research lab focused on making intelligence open and accessible to everyone. The Data Acquisition Team is responsible for sourcing and managing the data needed to train high-quality AI models.
As a Human Data Strategist, you will own the end-to-end pipelines that convert human expertise into training data for better AI models. You'll work closely with research teams to understand their needs, assess vendor capabilities, and ensure data quality and delivery.
Key responsibilities include:
- Collaborating with researchers to translate model capability requirements and development plans into concrete data acquisition strategies
- Evaluating and managing vendor relationships, defining task specifications, acceptance criteria, and holding providers accountable for quality, delivery, and cost
- Building and maintaining dashboards to track throughput, turnaround time, acceptance rates, rework, and cost per task; using data analytics to identify and resolve bottlenecks
- Maintaining alignment between data acquisition efforts and research priorities; surfacing vulnerabilities early and iterating on sourcing strategies based on feedback
- Closing the loop by tracking how data acquisitions translate into improvements in model capabilities
This role combines program management with hands-on technical work. You'll need organizational rigor, technical curiosity, and sound judgment about when to move quickly versus when to dig into details.
Requirements:
- 4+ years of experience delivering complex programs and managing priorities across evolving research needs
- Understanding of LLM training, evaluation, and prompt design, with ability to learn new research concepts quickly
- Proficiency in SQL for querying datasets and building dashboards
- Proficiency in Python for data analysis, API integrations, and workflow automation
- Strong written communication skills and sound stakeholder judgment
- Willingness to inspect individual tasks and datasets when aggregate metrics don't tell the full story
- High agency, bias toward action, and ability to drive impact in fast-paced, dynamic, ambiguous environments
- Plus: experience with annotation, expert data collection, human feedback, or AI evaluation
