SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Mistral is seeking a Finance Data Annotation Quality Analyst to join the Human Data Annotation team within the Science organisation. The role is hybrid, combining quality review and tooling responsibilities.
You will spend the majority of your time reviewing and auditing finance-domain annotations against rubrics to ensure data used for training and evaluating AI models meets the highest standards of quality and efficiency. The remainder of your time will be spent building, maintaining, and troubleshooting annotation operations.
Key Responsibilities:
- Collaborate with the technical team to review and audit annotations, clarify requirements, and share insights to surface systemic issues, edge cases, and gaps in guidelines
- Generate and validate high-quality finance-oriented data annotations yourself when needed, based on guidelines and continuous feedback, for AI model development and evaluation
- Build and maintain internal tools and automation that streamline annotation or review workflows, such as visualization dashboards, batch configuration scripts, and output management utilities
- Troubleshoot campaign issues related to tooling and guidelines for annotators
- Organize annotator training and coordinate feedback between annotators and reviewers
You will work closely with annotators, the technical program manager, and science stakeholders to refine guidelines and processes that shape how data is produced.
Requirements:
- Master's degree minimum in a Finance discipline
- At least 3 years of professional experience in the Financial domain
- Outstanding analytical and research skills
- Familiarity with programming languages and ability to read standard output formats such as .jsonl files
- Strong judgment with complex instructions, limited data, and multiple information sources; ability to surface edge cases, ambiguities, or gaps in guidelines
- Excellent communication, interpersonal, and organizational abilities
- Ability to adapt to dynamic environments and changing requirements
- High tolerance for repetitive, detail-oriented, high-volume review work with sustained focus and accuracy
- Professional proficiency in English with strong writing and comprehension skills
Nice to Have:
- Familiarity with rubric-based evaluation concepts, inter-annotator agreement, or quality measurement for human-labeled data
- Experience building annotation teams through interviews and training
- Proficiency in at least one programming language (e.g., Python)
- Prior experience in data annotation for AI/ML, especially LLM training (SFT, RLHF, preference data), evals/benchmarks, or agentic data
- Comfort working in Unix-like terminals and using git workflows
