SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Kpler is a global intelligence platform founded in 2014 that simplifies complex trade information and delivers actionable insights across commodities, energy, and maritime sectors. With over 850 experts across 69 countries, the company transforms intricate data into strategic intelligence for informed decision-making.
In this Senior Data Engineer role, you will architect, build, and optimize scalable data pipelines and stream-processing workflows using Python, Apache Spark, Kafka, and Airflow. You'll maintain and evolve data architecture across a polyglot backend microservices environment (Python/FastAPI, PHP, Java/Scala) to support continuous analytics delivery. A key focus is ensuring end-to-end data quality, freshness, and reliability through automated validation, monitoring frameworks, and strict SLA tracking.
You will provision and manage cloud-native data infrastructure on AWS using Terraform, Docker, and automated CI/CD practices. Cross-functional collaboration is essential—you'll work with Product Managers, SRE, Platform Engineers, and Data Scientists to translate complex domain needs into performant data products. You'll also drive operational and cost efficiency across data infrastructure, optimizing cloud resource usage and pipeline performance.
This is an individual contributor role focused on technical depth and architectural impact rather than people management.
**Requirements:**
- 6+ years of professional software or data engineering experience building complex, production-grade data processing systems
- Strong expertise in Python and hands-on experience with distributed data processing tools such as Apache Spark, Kafka, or Airflow
- Advanced knowledge of database design, SQL performance tuning, and data modeling across relational and NoSQL stores
- Proven experience managing cloud infrastructure on AWS, utilizing Terraform, Docker, and modern CI/CD pipelines
- Practical experience writing clean, maintainable code in polyglot environments (Python, Java/Scala, or PHP)
**Nice-to-haves:**
- Experience building RESTful APIs using Python frameworks like FastAPI
- Background handling high-throughput time-series, geospatial, or maritime logistics datasets
- Exposure to cloud FinOps, observability, and cost management tooling (e.g., Datadog, Astronomer)
