SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Wayve is building the leading AI platform for autonomous driving, using an end-to-end AI approach that enables vehicles to learn directly from real-world experience. The company is pioneering mapless, hardware-agnostic technology that adapts and generalizes at scale, partnering with global OEM manufacturers to enable continuous software evolution.
The Ingestion Systems team sits within Wayve's AI Platform organization and builds the data foundations behind the self-driving technology. Operating at petabyte scale, the team processes vast amounts of real-world driving data for annotation, model training, and evaluation, focusing on making these critical data flows reliable, efficient, and easy to operate.
In this role, you will:
- Debug and unblock failing ingestion pipelines, stalled jobs, and processing queues
- Investigate corrupt, malformed, or unexpected data from a variety of sources
- Optimize Spark jobs and batch-processing pipelines for throughput, reliability, and compute efficiency
- Work alongside permanent engineers to prioritize critical datasets and reduce operational toil
- Improve monitoring, retries, and failure recovery across multi-step ingestion workflows
- Build and maintain high-volume pipelines that process real-world driving data for annotation, data science, model training, and evaluation
- Design resilient systems that isolate problematic data rather than allowing a single bad segment to block wider workflows
- Manage orchestration across complex dependencies, queues, retries, and priority data flows
- Make pragmatic improvements to the performance, cost, and operability of Spark-based data platforms
- Handle ingestion of varied data formats from partners, suppliers, and third-party sources
Wayve operates a hybrid working model combining in-person collaboration in dedicated office spaces with focused remote work. Main hubs are in London, Sunnyvale, Yokohama, Herzliya, Vancouver, and Leonberg. The company is not a mature, fully-structured organization—much of how it works is still being written, and you'll help shape it. This suits people who want real ownership over their work.
REQUIREMENTS:
- Strong production experience with Apache Spark and Python
- Experience building, operating, or debugging large-scale ingestion, ETL, or distributed data-processing pipelines
- Comfort diagnosing production failures involving corrupt data, stalled queues, retries, and complex dependencies
- Understanding of how to optimize high-throughput systems for reliability, performance, and cost
- Ability to learn existing systems quickly, work independently, and deliver practical improvements in a fast-moving environment
- Advantageous: experience with Airflow, Flyte, Databricks, Delta Lake, Scala, or Java
Note: The posting encourages applications even if you don't meet every requirement, particularly if you're passionate about autonomy and keen to learn.
