SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Salary: USD 192,000 - 240,000 / annual
Datadog is seeking a Senior Software Engineer to work on Husky, a distributed, petabyte-scale columnar event store powering the company's Event Platform. Husky underpins dozens of Datadog's most popular products including Logs, RUM, APM, Cloud Network Monitoring, and Netflow, storing and querying massive volumes of event data at low cost with data fully queryable within seconds of arrival.
In this role, you will work hands-on with a distributed storage system at petabyte scale and help evolve the platform to support the next level of scale for the AI era. Key responsibilities include:
• Driving Husky adoption by migrating the storage format to open-source Parquet and reimagining compaction on a format-agnostic, DataFusion-based substrate
• Driving concrete cost-reduction initiatives: block-level inverted index compression, combining the database with external objects at the row level, and storage tiering/caching
• Scaling the solution to enable customers to manage event versioning at scale using temporal-database techniques
• Playing an influential role in operating and deciding how to best operate the service going forward
• Working across the write path with the Ingestion team and toward a cleaner Iceberg-based handoff to the Lakehouse team for batch and analytical workloads
You will partner closely with Husky Ingestion, Metadata, and Platform teams, and increasingly with Lakehouse on the Iceberg-compatible boundary. You will also work with the open-source community to build the next generation of reusable software solutions that can be operated at scale.
Datadog's culture is engineering-driven, bottom-up, and high-agency. The company emphasizes blameless culture, open-source advocacy, and flexible remote work while valuing in-person collaboration across offices.
Requirements:
• Enthusiastic and experienced with large-scale distributed systems and/or storage engines
• Comfortable with improving and operating large distributed systems that customers rely on for critical business functions
• Strong fundamentals in columnar/file storage formats (Parquet, ORC, Arrow) or comparable systems
Bonus qualifications:
• Experience with Go, Java, or Rust
• Experience with compaction or compression/encoding schemes (RLE, dictionary, delta, gorilla)
• Experience with object storage (S3/GCS/Azure Blob) at scale, including caching and cost optimization
• Experience with Kafka and stream processing
• Experience with Kubernetes
