SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Lighthouse is an AI-powered commercial strategy platform for hospitality, processing ~200 million requests daily across 200+ data sources. The company has raised $370M in Series C funding, generates over $100M in ARR, and serves 85,000+ users including all top 15 global hotel chains.
You will join the Crawler Engineering team as a Backend Engineer focused on data acquisition. This is a hands-on building role where you'll design and develop high-performance systems—primarily in Rust—that collect billions of data points from the web. The work bridges backend engineering with web infrastructure: HTTP stacks, proxies, and browser tooling. When data sources block an approach, you'll investigate what's happening at the protocol level and find alternative solutions.
Key responsibilities:
- Design, build, and improve backend systems for data acquisition handling ~200 million requests daily
- Work on the in-house proxy layer and browser tooling: request routing, connection handling, and protocol-level behavior
- Investigate complex crawler problems, test alternative approaches when blocked, and turn working solutions into robust, reusable systems
- Build clear interfaces and guardrails so other teams and AI agents can reliably use your infrastructure
- Own code quality including tests, observability, and maintainability (including code written with AI assistance)
- Collaborate with crawler engineers, DevOps, and Security teams to ship secure, scalable solutions
- Help make infrastructure "agent-friendly" for AI-driven development
The Crawler Engineering team is responsible for ensuring customers receive fresh, high-quality rates, ranking, and supply data. The team maintains reliable, efficient crawling infrastructure that adapts to a constantly changing web.
Benefits include flexible working (home or global offices), flexible time off, career development workshops and training, mobility budget or company car, competitive compensation, home office allowance, lunch and eco vouchers, comprehensive health insurance, pension funding, and referral bonuses.
Requirements:
- Hands-on experience building, deploying, and running production backend services in a cloud environment
- Experience with systems languages such as Rust or C/C++, OR strong track record in another backend language with genuine drive to go deep in Rust
- Solid grasp of web fundamentals: HTTP, TCP/IP, TLS, and proxies
- R&D mindset: curiosity about how browsers and network stacks work internally, comfort with ambiguity, preference for hypothesis-driven validation
- Strong analytical and debugging skills, comfortable with packet captures, profilers, and reading source code
- Clear communication, openness to feedback, strong teamwork
- Fluency in English
Technologies: Rust, TypeScript/Node.js, Python, Google Cloud, Kubernetes, Prometheus, Grafana, Claude Code.
