SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
DataHub is an AI & Data Context Platform used by over 3,000 enterprises including Apple, CVS Health, Netflix, and Visa. The company provides a metadata graph platform with AI-powered discovery, observability, and governance capabilities for data and AI assets.
You will build and evolve DataHub's Python-based metadata ingestion framework, which powers metadata collection across the modern data stack. This role focuses on expanding the foundation with deeper integrations, cloud-native execution, and AI-assisted capabilities.
Key responsibilities include:
- Build and evolve the Python-based metadata ingestion framework to enable rich ingestion of usage statistics, lineage, schemas, operational metadata, quality signals, and business context
- Design and ship high-quality connectors for modern data and AI platforms including Snowflake, Redshift, Kafka, Databricks, and emerging infrastructure
- Make ingestion cloud-native: scalable, resilient, observable, and easy to deploy and operate across customer environments
- Apply AI to create smarter connectors, including metadata inference, automated enrichment, and intelligent ingestion workflows
- Solve challenging distributed-systems problems involving large-scale metadata extraction, incremental updates, reliability, and performance
- Lead technical direction for a small engineering pod, mentor junior engineers, and raise the bar for engineering quality, design, and execution
This is a senior individual contributor role with mentorship and technical leadership responsibilities for a small team. You'll work on infrastructure that powers metadata collection at scale for enterprise customers.
