SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Kpler is a global cargo intelligence platform serving the commodities, energy, and maritime sectors. Founded in 2014, the company employs over 850 experts across 69 countries, transforming complex trade data into actionable insights.
As a Senior Software Engineer within the Commodities tribe, you will lead the end-to-end integration of US inland waterway barge data into Kpler's global platform. Operating under a "you build it, you run it" philosophy, you will own the full engineering lifecycle from external feed ingestion and domain modeling through streaming, entity resolution, and customer-facing delivery.
Key responsibilities include:
- Lead end-to-end integration architecture, driving the full lifecycle from external feed ingestion through domain modeling, streaming, entity resolution, and UI distribution
- Design robust ingestion and entity-resolution systems, mapping incoming barge data onto Kpler reference data (vessels, zones, installations, products) using Elasticsearch and PostgreSQL matching patterns
- Build scalable Kafka streaming pipelines with event-driven contracts, Avro schemas, deduplication mechanisms, and dead-letter queue replays using Python and Scala
- Deliver integrated barge data across customer surfaces including Elasticsearch read models, external APIs, data warehouse feeds, and the Kpler Terminal (TypeScript/Vue)
- Champion production quality and observability by maintaining full operational ownership, setting SLOs, participating in on-call rotations, and driving post-incident improvements
- Ensure data integrity and analyst alignment by validating external feeds and integrating human-in-the-loop analyst workflows
- Mentor crew members, document architectural designs, and contribute to decomposing legacy systems into event-driven services
Requirements:
Must-haves:
- ~5+ years of production data engineering experience designing, operating, and maintaining data-intensive systems with full ownership
- Advanced Python and SQL expertise with proven ability to build clean, well-bounded services in large, complex, evolving codebases
- Hands-on production experience with Kafka streaming, including schema management (Avro/Schema Registry), deduplication, and replay patterns
- Demonstrated background integrating external data feeds, managing schema mapping, and building data quality monitoring frameworks
- Direct experience managing production observability, writing documentation, and resolving incidents in distributed system environments
Nice-to-haves:
- Functional programming and Scala experience, or track record of rapidly mastering new languages
- Geospatial data tools (PostGIS) or exposure to maritime, logistics, or commodities domains
- Modern data stack exposure (Elasticsearch, Airflow, Snowflake/lakehouse formats, Kubernetes, GitOps)
- Working knowledge of TypeScript or Vue for user-facing terminal collaboration
