SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Lighthouse is an AI-powered commercial intelligence platform for hospitality, serving tens of thousands of hotels across 185 countries. The company has raised $370M in Series C funding and generates over $100M in ARR, with 850+ employees across 35 countries.
You will join the Crawler Engineering team as a Backend Engineer focused on data acquisition. This is a hands-on building role where you'll design and develop high-performance systems—primarily in Rust—that collect billions of data points from the web. The work bridges backend engineering with web infrastructure: HTTP stacks, proxies, and browser tooling. When data sources block an approach, you'll investigate what's happening at the protocol level and find alternative solutions.
Key responsibilities:
- Design, build, and improve backend systems for data acquisition handling close to 200 million requests per day
- Work on the in-house proxy layer and browser tooling, including request routing, connection handling, and protocol-level behavior
- Investigate complex crawler problems, test alternative approaches when blocked, and turn working solutions into robust, reusable systems
- Build clear interfaces and guardrails so other teams and AI agents can reliably use your infrastructure
- Own code quality including tests, observability, and maintainability—including code written with AI assistance
- Collaborate with crawler engineers, DevOps, and Security teams to ship secure, scalable solutions
The Crawler Engineering team maintains the infrastructure that delivers fresh, high-quality rates, ranking, and supply data to Lighthouse customers. The team manages 200+ data sources and carries responsibility for keeping the crawling infrastructure reliable, efficient, and ahead of a constantly evolving web.
The role offers flexible working (remote or office-based), flexible time off, health insurance (100% Alan coverage for employee, spouse, and dependents), subsidized ClassPass (up to 80%), food/nursery benefits (€160/month), and access to earned salary in advance through Payflow App.
REQUIREMENTS:
- Hands-on experience building, deploying, and running production backend services in a cloud environment
- Experience with a systems language such as Rust or C/C++, OR strong track record in another backend language with genuine drive to go deep in Rust
- Solid grasp of web fundamentals: HTTP, TCP/IP, TLS, and proxies
- R&D mindset: curiosity about how browsers and network stacks work internally, comfort with ambiguity, preference for hypothesis-driven validation
- Strong analytical and debugging skills, comfortable with packet captures, profilers, and reading others' source code
- Clear communication, openness to feedback, strong teamwork
- Fluency in English
Technologies: Rust, TypeScript/Node.js, Python, Google Cloud, Kubernetes, Prometheus, Grafana, Claude Code.
