SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Gallatin is rebuilding defense logistics for the US military and allied forces using AI-first software to modernize logistics operations. The Data Platform team connects fragmented data sources (legacy systems, spreadsheets, sensor feeds) and serves curated, semantically modeled data to Navigator and AI agents.
You will design and ship containerized services that create and manage external data connections, operate an open lakehouse, and power transformations from raw source data to trusted entities. You'll build fullstack applications (TypeScript, Vue, Python) that let logistics subject-matter experts and analysts connect sources, map fields, and test pipeline changes independently.
The platform runs across commercial cloud, government cloud, single-tenant high-side enclaves, and air-gapped appliances at forward sites—requiring Kubernetes-native, open-source components and data movement tolerant of intermittent, disconnected networks.
Key responsibilities:
- Design and operate containerized services for external data connections (databases, REST/SOAP APIs, file drops, message buses, legacy systems)
- Own the open lakehouse: object storage, Apache Iceberg tables, REST catalog with data versioning, and federated query through Trino
- Build batch and streaming ingestion with change data capture, designed for DDIL (disconnected, degraded, intermittent, limited) networks
- Operate the platform on Kubernetes across commercial cloud, government cloud, and air-gapped edge appliances using hardened, FIPS-validated images
- Build transformation pipelines through raw, cleaned, and curated layers into semantic entity models
- Implement entity resolution and data fusion so each asset resolves to one record across sources
- Structure unstructured inputs (documents, slides, spreadsheets) into typed, relational records
- Build lineage, provenance, data quality checks, and confidence labeling into every pipeline
- Build fullstack applications that let SMEs connect sources, map fields, and compose pipelines with low-code workflows
- Provide sandboxes for branching data, testing pipeline changes, comparing results, and promoting to production
- Build documented APIs for Navigator features and AI agents
- Partner with Product, AI/ML, Frontend, and Infrastructure teams on RFCs, data models, and production code
- Work with mission partners and field teams to translate operational constraints into platform requirements
- Mentor engineers as the data team grows
Requirements:
- 8+ years of professional data engineering experience, including hands-on design and operation of production data pipelines and ownership of a data platform or lakehouse
- Hands-on production experience with an open table format (Apache Iceberg preferred; Delta or Hudi acceptable) and a distributed query engine (Trino, Presto, or Spark SQL), including partitioning, compaction, and schema evolution
- Demonstrated experience with streaming and batch data pipeline technologies (Kafka, NiFi, Debezium, Spark, or similar)
- Strong proficiency in Python and SQL, with experience integrating and validating data from multiple heterogeneous sources (relational and NoSQL databases, APIs, files, and streams)
- Strong data modeling skills for both analytical and operational workloads; experience designing entity models for complex real-world domains
- Production orchestration experience (Airflow, Dagster, Argo Workflows, or similar)
- Production experience writing services in Python and/or TypeScript, plus enough front-end skill (Vue, React, or similar) to ship a usable internal tool end to end
- Comfortable designing and documenting RESTful APIs that other teams build on
- Strong Kubernetes and Docker experience: packaging, deploying, and operating stateful data services on Kubernetes; ability to debug networking, storage, and namespace issues
- Comfortable building on open-source tooling for self-hosted and air-gapped deployments
- Experience with AWS data services (S3, Glue, Aurora) or Azure equivalents and containerized environments (Kubernetes) delivered through DevSecOps/CI-CD pipelines
- Demonstrated ability to work independently against well-defined tasking with minimal oversight
- Effective communication with both technical and non-technical stakeholders, including data providers and consumers
- Clear communicator across disciplines (RFCs, data models, weekly demos)
- Always seeking to learn and stay current with industry trends and tools
Bonus qualifications:
- Active SECRET or TOP SECRET clearance
- Data catalog, lineage, and governance experience (Iceberg REST catalogs, Nessie, Polaris, OpenLineage, DataHub, or similar)
- Data versioning and branching (Nessie, lakeFS) for sandboxed pipeline testing
- Apache NiFi or other low-code/no-code pipeline tools, including custom processors
- Entity resolution, semantic data fusion, knowledge graphs, or ontology modeling
- Shipping data workloads into air-gapped or classified environments
- Palantir Foundry, Apollo, or ontology experience
- Data infrastructure for AI: metadata and retrieval that make agents more accurate and explainable
- Operations research, optimization, or constraint-solver background
- Background in defense logistics, sustainment, or supply chain systems
- Contributions to open-source data projects
Note: This position requires the ability to obtain and maintain relevant security clearances and work in classified environments when necessary.
