SlipstreamJobsFresh Startup & VC-Backed Jobs

Principal Software Engineer, Data Infrastructure

Tally - San Francisco, CA, United States - Hybrid

Apply on the company site

SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.

Salary: USD 197,000 - 230,000 / annual

Happen Bank (formerly LendingClub) is building a modern enterprise data and AI platform on Databricks. As a Principal Software Engineer for Data Infrastructure, you will own the technical architecture and development of platform frameworks and infrastructure that enable downstream data pipelines to operate faster, safer, and more consistently. This is a framework and platform engineering role focused on enabling teams through configuration rather than custom code. You will define the technical vision for a declarative, configuration-driven ingestion framework that brings data from relational databases and object-store files into the raw data layer. You'll lead the design and implementation of reusable, versioned libraries supporting policy-driven data protection with configurable rules, governed access, consistent outputs, and secure-by-default failure behavior. Key responsibilities include establishing infrastructure-as-code practices for safe deployment, rollback, and lifecycle management; driving platform reliability through monitoring, alerting, reconciliation, and data validation; conducting structured proofs of concept with measurable success criteria; translating ambiguous technical challenges into clear requirements and phased delivery plans; and mentoring senior and staff engineers while establishing best practices across shared platform capabilities. You will partner cross-functionally with Data Engineering, Data Governance, Information Security, DBAs, and source-system owners to deliver scalable long-term capabilities. The role requires navigating ambiguous technical challenges, evaluating tradeoffs thoughtfully, and taking ownership of solutions that improve engineering effectiveness across teams. Work location is San Francisco with a hybrid model (in-office Tuesdays, Wednesdays, Thursdays). In-person attendance is essential. Local Pacific Time hours required, with flexibility across time zones as needed. Travel as needed to Happen Bank offices. REQUIREMENTS: - 10+ years of software or data platform engineering experience, including designing and delivering platform-level frameworks, libraries, or infrastructure used across multiple engineering teams - Bachelor's degree or higher, or equivalent combination of education and experience - Strong experience with infrastructure as code, including Terraform and CI/CD pipelines for data platforms - Deep experience building ingestion capabilities across diverse data sources (relational databases, files, APIs, custom systems) with expertise in incremental processing, change data capture (CDC), schema evolution, and protecting production source systems - Advanced programming skills in Python, SQL, and a JVM language (Scala or Java), with experience developing Spark UDFs and reusable libraries using strong testing, packaging, and versioning practices - Deep hands-on expertise with Databricks (Delta Lake, Unity Catalog, Auto Loader, workflows, compute configuration, governance) or equivalent data platform experience - Strong knowledge of security and privacy engineering for sensitive data (encryption, key management, tokenization, format-preserving techniques, RBAC, audit logging) - Experience designing and operating large-scale data systems processing billions of rows, including performance testing, cost optimization, monitoring, and recovery strategies - Excellent written and verbal communication skills, including ability to develop clear technical designs, architecture decision records, interface contracts, and documentation while influencing senior stakeholders without direct authority - Deep experience applying AI tools to complex engineering work (architecture exploration, development, testing, problem-solving) NICE TO HAVE: - Experience in banking, lending, or regulated environments with model risk management, SOX, or similar audit requirements - Experience with data quality, lineage, catalog tooling, and data contract approaches - Prior experience as Principal Engineer, Staff Engineer, or technical lead for a platform engineering team - Experience with Databricks Asset Bundles or equivalent deployment tooling

Similar roles