SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Mistral is seeking a Research Engineer to join the Latency & Throughput team, pushing the frontier of LLM inference by researching and prototyping techniques that reduce latency and increase throughput. The role sits at the intersection of research and production engineering, requiring you to take ideas from recent literature, form hypotheses, run experiments, and ship validated techniques into the production serving stack.
You will be more science-focused than a pure engineer, expected to generate new ideas, keep up with state-of-the-art developments, and implement proof-of-concepts that bridge academic results and production impact. Some techniques explored will be lossy or require integration into training—retraining models, tuning architectures, or training auxiliary models to improve inference.
Key responsibilities include:
- Research and prototype inference techniques such as speculative decoding, quantization, KV compression, watermarking, and new model architectures by reading recent literature and forming testable hypotheses
- Design and run experiments to validate or refute ideas, iterating quickly on small-scale setups before proposing production integration, demonstrating clear latency or throughput gains without unacceptable quality loss
- Collaborate with the Inference Foundation team to integrate validated techniques into the vLLM-based serving stack
- Partner with pretraining and RL teams when techniques require co-training, retraining, or training auxiliary models
- Build tooling and validation platforms to facilitate AI-driven inference research
You will work closely with inference engineers who own the production serving stack and research scientists focused on paper-level contributions. Success is measured by demonstrated latency or throughput gains, first at small scale, then adopted in production.
Mistral provides full-stack AI solutions from frontier models to developer tools, applications, and compute, partnering with enterprises across finance, manufacturing, defense, healthcare, and the public sector.
REQUIREMENTS:
- Strong ML background with solid knowledge of LLM training techniques (architecture, tokenization, fine-tuning, evaluation)
- Hands-on experience with Python and PyTorch; ability to write clean, experimental code and iterate fast (JAX acceptable with strong Python proficiency)
- Research experience of some form: thesis, publications, dedicated research project, or efficiency-oriented work in a lab or industry research setting (PhD not required but genuine research depth expected)
- Track record of conducting experiments: forming hypotheses, designing controlled comparisons, drawing clear conclusions from results
- Familiarity with state of the art in LLM inference—knowledge of what has been tried, what works, and where open problems are
- Self-starter, low-ego, collaborative; comfortable with ambiguity and negative results
- C++ and Rust not required; CUDA and GPU knowledge expected but hands-on experience not required
- Inference or serving experience is a plus (prior work with vLLM, production LLM deployments, or serving systems helpful but not required)
