SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Kpler is a global market intelligence platform founded in 2014 that simplifies complex trade information across commodities, energy, and maritime sectors. The company employs over 850 experts from 69 countries and transforms intricate data into actionable strategies for clients worldwide.
As a Senior Data Engineer on the new Marine Insurance initiative, you will be a technical leader responsible for designing and delivering end-to-end data pipelines and cloud infrastructure that power Kpler's global platform. You will own production-grade systems from architecture through deployment, monitoring, and maintenance, handling large-scale data ingestion, processing, and serving to global customers.
Key responsibilities include:
- Design and own end-to-end data pipelines: Build, deploy, and operate production-grade data pipelines and infrastructure from initial architecture through deployment, monitoring, and maintenance.
- Deploy containerized AWS services: Develop and containerize applications within AWS using Python, Docker, Kubernetes, Airflow, and Infrastructure as Code (Terraform).
- Ensure data integrity and event streaming: Architect robust PostgreSQL database structures and event-streaming flows (Kafka/Flink) to guarantee seamless data flow and integrity.
- Set technical direction and standards: Lead complex pipeline design, peer reviews, and logging, monitoring, and alerting frameworks to maintain pipeline resilience.
- Partner cross-functionally and mentor peers: Collaborate closely with engineers, product managers, and data scientists while mentoring engineers to elevate team craft and technical development.
- Evaluate emerging data tools: Research and implement modern data infrastructure improvements and frameworks to continuously elevate Kpler's platform capabilities.
Requirements:
Must-haves:
- 5+ years of experience designing, operating, and owning production-grade data pipelines and infrastructure at scale
- Strong Python skills with hands-on experience in Docker, Kubernetes, AWS, Airflow, and relational databases like PostgreSQL
- Practical experience with event streaming (Kafka/Flink), Infrastructure as Code (Terraform), CI/CD practices, and production logging/monitoring systems
- Demonstrated ability to lead architectural discussions, mentor engineers, and leverage AI-assisted coding tools (Copilot, Cursor, Claude) with strong technical judgment
- Experience working in Agile environments with strong collaboration skills and fluency in written and spoken English
Nice-to-haves:
- Experience building large-scale data processing systems or analytics platforms
- Track record of leading technical initiatives across multiple teams
- Familiarity with commodities, shipping, energy, or marine insurance market data
