SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Salary: USD 240,000 - 300,000 / annual
SecurityScorecard is the global leader in cybersecurity ratings, continuously rating over 12 million companies across 64 countries. The company's patented rating technology is used by 25,000+ organizations for self-monitoring, third-party risk management, board reporting, and cyber insurance underwriting.
You will serve as Principal Software Architect for the data platform, an individual contributor role reporting to the Chief Architect. This is a high-impact position where you own end-to-end system design for a mission-critical data platform that ingests internet-scale measurement data, processes it through streaming, microbatch, and batch paths, stores it affordably at scale, and serves analytics in real time to customers. The data itself is the product—not a byproduct—which raises the stakes on correctness, quality, and lineage.
Key responsibilities include:
- Own system design for the data platform from ingestion through serving layer
- Define service boundaries and data contracts between producers and consumers, including schema ownership and compatibility rules
- Design the lakehouse architecture: table format, partitioning, schema evolution, compaction, and metadata management at scale
- Architect the analytical serving layer for three conflicting workload classes (low-latency product queries, ad-hoc internal analytics, bulk external feeds) with isolation guarantees
- Set direction on languages and frameworks in the data stack
- Engineer data quality and observability into the platform: validation, quarantine paths, freshness/completeness SLOs, drift detection, and auto-generated lineage
- Design for correctness and reproducibility in the ratings pipeline, including backfills and historical restatement
- Write Technical Design Reviews (TDRs), design docs, and standards that guide data architecture across teams
- Review TDRs from across engineering, providing substantive feedback on architecture and risk
- Partner with AI and Front-end Architects on data access patterns; mentor senior and staff engineers
The tech stack includes Kafka for event streaming, Flink (Java) for stream processing, Spark for batch/microbatch, ClickHouse for analytical serving, PostgreSQL for operational storage, Node.js/TypeScript for the platform, React for frontend, and AWS/Kubernetes for infrastructure. Some high-performance components are written in C++.
You will operate as a senior individual contributor, driving outcomes through prototyping and technical credibility rather than people management. You'll be opinionated about data architecture and persuasive in earning buy-in from skeptical engineers through visible reasoning.
**Requirements:**
- 10+ years of software or data engineering experience, including significant time architecting large-scale data platforms
- Deep expertise in stream and batch processing at scale: Kafka, Flink, Spark or equivalents, with clear judgment about workload placement
- Strong Python and PySpark; solid Java for Flink stream processing; enough Scala to read and reason about Spark codebases
- Hands-on production experience designing lakehouse storage: columnar formats (Parquet), open table formats (Iceberg), partitioning, compaction, schema evolution
- Experience architecting OLAP and analytical serving layers (ClickHouse, Druid, Pinot, BigQuery, Snowflake or similar)
- Strong distributed systems fundamentals applied to data: exactly-once vs. at-least-once semantics, ordering, backpressure, late/out-of-order data, pipeline failure modes
- Track record of building data quality, contracts, and observability as engineered system properties (assertions, schema enforcement, lineage in code)
- Experience owning large-scale data migrations, preserving history and correctness through cutover
- Proven influence without authority: presenting technical direction to skeptical engineers and earning genuine buy-in; giving rigorous design review feedback on systems you didn't build
- Strong technical writing and mentorship: TDRs, design docs, decision records that teams can act on independently; history of raising technical bar
- Comfort operating as a senior individual contributor, driving outcomes through prototyping and technical credibility
**Preferred Qualifications:**
- Experience building data layers for ML/LLM systems: feature stores, vector stores, retrieval pipelines
- Internet-scale scan, telemetry, observability data experience, or cybersecurity industry background
- Track record of reducing platform costs at scale through storage tiering, query governance, or compute right-sizing
