SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Perplexity is building advanced search and answer systems powered by AI. This role sits at the intersection of data analysis and machine learning engineering, focused on improving the company's crawling and storage infrastructure that processes trillions of URLs across the internet.
You will work on a complex multi-stage system (URL discovery, crawling, parsing, indexing) where each stage presents opportunities for optimization and contains intricate bugs. Your responsibilities will include:
- Identifying and diagnosing quality issues within the crawling pipeline
- Training small machine learning models to optimize specific pipeline aspects (e.g., parsing quality)
- Building datasets for model training, including LLM-as-a-judge labeling pipelines
- Improving page selection algorithms for indexing
- Designing and analyzing experiments to validate improvements
This is a hands-on technical role requiring both analytical rigor and engineering capability. You'll design metrics from scratch, build data pipelines, and directly contribute to improving search quality and answer accuracy.
Qualifications:
- 4+ years of experience as a data analyst, ML engineer, or related role
- Strong coding skills at the level of a mid-level backend engineer; ability to write production-grade code
- Demonstrated experience designing metrics from scratch
- Track record training ML models that shipped to production with measurable metric improvements
- Nice to have: direct experience working on web crawling or indexing pipelines
