SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Distributed Spectrum builds systems for radio spectrum intelligence, collecting and processing radio data globally using neural networks and custom hardware. The company has developed a proprietary stack including novel embedding model architectures, custom GPU kernels, and specialized infrastructure for signal processing.
You will join as Senior Data Engineer and own the architecture of a data lakehouse platform that processes continuous, high-rate radio data streams from sensors worldwide. This platform serves three critical functions: enabling researchers to train models, supporting production inference systems, and providing real-time context retrieval for agents.
Key responsibilities include:
- Architecting and building the data lakehouse: ingestion pipelines, open table formats (Iceberg, Delta Lake, Hudi), partitioning and compaction strategies, cataloging, and governance
- Designing and operating vector database infrastructure for embedding storage, similarity search, and retrieval at scale
- Building large-scale data management systems for lifecycle, retention, lineage, dataset versioning for reproducible training, quality monitoring, and cost management across petabyte-scale storage
- Designing data platform capabilities for ML use cases: feature and embedding pipelines, training-set assembly, evaluation datasets, and low-latency retrieval for agents
- Establishing schema and data-contract standards across teams and building self-serve tooling for researchers and engineers
- Owning reliability and performance of production data pipelines, including observability and failure handling
- Mentoring engineers and leading technical design across the data domain
The role emphasizes high ownership, technical rigor, and collaboration within a talent-dense team with deep expertise in probabilistic ML, accelerated computing, embedded systems, and signal theory.
REQUIREMENTS:
- 4–6+ years of software/data engineering experience, including several years designing and operating large-scale data platforms in production
- Deep experience with lakehouse architectures and technologies (Apache Iceberg, Delta Lake, Hudi, Parquet, Spark, Flink, Trino, DuckDB, or similar)
- Hands-on experience with vector databases and embedding retrieval (pgvector, Milvus, Qdrant, Weaviate, Pinecone, LanceDB, FAISS-based systems), including indexing trade-offs and scaling
- Strong experience with AWS or another major cloud provider (object storage, managed data services, compute, cost optimization at scale)
- Fluency in Python and SQL; experience with orchestration tools (Airflow, Dagster, Prefect, Step Functions, etc.)
- Demonstrated experience designing data architectures for ML/AI workloads: training data pipelines, feature stores, dataset versioning, or retrieval systems
- Strong grasp of data modeling, consistency, and batch vs. streaming trade-offs
NICE TO HAVE:
- Experience with streaming ingestion at high volume (Kafka, Kinesis, Pulsar)
- Experience with time-series, geospatial, or signal/sensor data
- Familiarity with data governance and security in regulated environments
- Experience with Rust, Go, or C++ for performance-critical data paths
NOTE: This role requires U.S. citizenship, lawful permanent residency, or eligibility to obtain ITAR authorization from the U.S. Department of State due to export control regulations.
