SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Salary: USD 160,000 - 200,000 / annual
People Data Labs is a provider of people and company data, integrating thousands of compliantly sourced datasets into a developer-friendly source of truth. Leading companies use PDL's workforce data to enrich recruiting platforms, power AI models, create custom audiences, and more.
You will join the Data Engineering & Acquisition Team to accelerate efforts building standalone data products that enable data teams and independent developers to create innovative solutions at scale. Working with a team focused on continuous improvement of existing datasets and pursuing new ones, you'll have high autonomy and direct impact.
Key responsibilities:
- Contribute to architecture and improvement of the data acquisition and processing platform, increasing reliability, throughput, and observability
- Use and develop web crawling technologies to capture and catalog data on the internet
- Build, operate, and evolve large-scale distributed systems that collect, process, and deliver data from across the web
- Design and develop backend services managing distributed job orchestration, data pipelines, and large-scale asynchronous workloads
- Structure and model captured data, ensuring high quality and consistency across datasets
- Continuously improve speed, scalability, and fault-tolerance of ingestion systems
- Partner with data product and engineering teams to design and implement new data products, and enhance existing ones
- Learn domain-specific knowledge in web crawling and data acquisition with mentorship from experienced teammates
The company values thoughtful, quirky engineers willing to fearlessly try new things. Failure is embraced as long as learning and growth follow. You'll work in a fast-paced, remote-first environment with asynchronous collaboration.
REQUIREMENTS:
- 7+ years of professional experience building or operating backend or infrastructure systems at scale
- Solid programming experience in Python, Go, Rust, or similar, including async/await, coroutines, or concurrency frameworks
- Strong grasp of software architecture and backend fundamentals; ability to reason about concurrency, scalability, and fault tolerance
- Solid understanding of browser rendering pipeline, web application architecture (auth, cookies, HTTP request/response)
- Familiarity with network architecture and debugging (HTTP, DNS, proxies, packet capture and analysis)
- Solid understanding of distributed systems concepts: parallelism, asynchronous programming, backpressure, and message-driven design
- Experience designing or maintaining resilient data ingestion, API integration, or ETL systems
- Proficiency with Linux/Unix command-line tools and system resource management
- Familiarity with message queues, orchestration, and distributed task systems (Kafka, SQS, Airflow, etc.)
- Experience evaluating and monitoring data quality, ensuring consistency, completeness, and reliability across releases
- Ability to work independently, proactively unblock yourself, and collaborate asynchronously
- Clear and thoughtful written communication (Slack, docs, design proposals)
- Ability to write and maintain technical design documents including pipeline design, schema design, and data flow diagrams
- Ability to scope and break down complex projects into deliverable milestones, communicating progress, risks, and blockers effectively
- Balance pragmatism with craftsmanship, shipping reliable systems while continuously improving them
NICE TO HAVES:
- Degree in a quantitative field such as computer science, mathematics, or engineering
- Experience as a Red Teamer
- Experience working on large-scale data ingestion, crawling, or indexing systems
- Experience with Apache Spark, Databricks, or other distributed data platforms
- Experience with streaming data systems (Kafka, Pub/Sub, Spark Streaming, etc.)
- Proficiency with SQL and data warehousing (Snowflake, Redshift, BigQuery, or similar)
- Experience with cloud platforms (AWS preferred, GCP or Azure also great)
- Understanding of modern data storage and design patterns (parquet, Delta Lake, partitioning, incremental updates)
- Experience building and maintaining data pipelines on modern big-data or cloud platforms (Databricks, Spark, or equivalent)
