SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Salary: USD 200,000 - 280,000 / annual
Glean is a Work AI platform that helps enterprises work smarter with AI. The company has evolved from enterprise search into a full-scale Work AI ecosystem, powering intelligent search, AI assistants, and scalable AI agents on a secure, open platform. With over 100 enterprise SaaS connectors, flexible LLM choice, and robust APIs, Glean enables organizations to govern, scale, and customize AI across their business without vendor lock-in.
This role sits at the intersection of applied research, quality, and product engineering, focused on improving the reliability and usefulness of Glean's enterprise AI assistants and autonomous agents. The core challenge is not writing code—it's knowing what good looks like, measuring it rigorously, and systematically closing the gap between current and desired performance.
You will work on quality improvements across Glean's models and agent harnesses, including:
- Using eval results and production traces to improve the entire stack (agent harness, model routing, prompts, retrieval) while balancing quality against latency and cost.
- Post-training models with supervised fine-tuning (SFT) and preference- or reward-based optimization techniques (DPO, PPO, GRPO), including generating training data.
- Building RL environments for simulating enterprise workflows, with relevant evalsets, harness support, and LLM-based verifiers for hill-climbing and benchmarking.
- Improving the quality of representative evalsets, LLM judges, and agentic verifiers; calibrating them against human graders so scores can be trusted in release decisions and model training.
- Collaborating with product, platform, and research teams to land quality improvements in production.
The role is hybrid, requiring 4 days per week in the Mountain View or San Francisco office.
REQUIREMENTS:
- 5+ years of industry experience in AI or Machine Learning Engineering.
- Hands-on experience post-training LLMs, including adapting open-weight models to specialized tasks.
- Experience building evals for AI assistant and agent systems: generating evalsets, building LLM-as-a-judge verifiers, calibrating offline and online signals, and good judgment about when a metric reflects genuine quality improvements.
- Strong coding skills in Python and/or Go (TypeScript a plus); proficiency in your ML framework of choice; track record of shipping production ML systems.
- Product-minded; able to turn vague problems into concrete experiments; thrive in tight-knit, cross-functional teams.
