SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Cantina Labs is a social AI company developing advanced real-time models for generative video and character creation. The Member of Technical Staff, Video Data & Model Evaluation role is based in Singapore and focuses on shaping data and evaluation strategies for video generation models.
You will own the complete data lifecycle for video generation, from defining data requirements and annotation standards to building high-quality datasets, evaluating model outputs, and translating findings into actionable recommendations for model improvement. You will collaborate closely with research scientists, machine learning engineers, product teams, and external data partners to strengthen the data–model–evaluation feedback loop.
Key responsibilities include:
- Define quality standards for video generation across aesthetics, realism, composition, motion, temporal consistency, instruction following, subject consistency, editing preservation, and text rendering
- Evaluate model iterations, analyze failure patterns, and develop data, training, and prompting strategies
- Own end-to-end training data projects including sourcing, construction, annotation, captioning, quality control, and delivery
- Develop labeling systems supporting dataset segmentation, coverage analysis, quality filtering, training-data mixtures, and model performance attribution
- Design annotation and captioning standards for video generation, editing, and multi-reference tasks
- Create and evaluate synthetic-data workflows based on stability, usability, quality, and training value
- Develop automated annotation and filtering workflows using vision-language models, system prompts, and quality-scoring models
- Build structured model evaluation processes covering test-set design, evaluation criteria, failure attribution, and reporting
- Establish feedback loops connecting model evaluation and negative-case analysis with data strategy and model iteration
- Train and calibrate annotation teams and external data partners, conduct quality audits, and improve delivery standards
- Partner with research, engineering, product, and creative teams to execute data initiatives and identify future model capabilities
Requirements:
- Hands-on experience in data strategy, dataset development, annotation, or model evaluation for generative image, video, or multimodal models
- Experience supporting model development across continued training, supervised fine-tuning, post-training, or evaluation
- Strong understanding of generative media tasks, including image and video generation, multimodal generation, and instruction-based editing
- Experience developing labeling taxonomies, annotation guidelines, captioning standards, quality thresholds, and evaluation frameworks
- Strong visual judgment and ability to translate subjective quality expectations into measurable data and evaluation standards
- Experience managing large-scale data-production or annotation projects and collaborating with research, engineering, creative, and external partner teams
- Familiarity with automated captioning, synthetic-data generation, vision-language models, prompt design, Python, SQL, or other data and workflow automation tools (preferred)
