Principal Software Engineer, Data Infrastructure

LendingClub•San Francisco, CA

3d•$197,000 - $230,000•Hybrid

About The Position

Happen Bank is building a modern enterprise data and AI platform on Databricks. This principal-level platform engineering role owns the architecture and development of frameworks and infrastructure that enable data engineering, data science, and analytics teams to build downstream pipelines more quickly, safely, and consistently. The work includes configuration-driven ingestion, secure source-system connectivity, reusable data-protection libraries for anonymization and tokenization, and infrastructure-as-code foundations. The role focuses on shared platform capabilities rather than delivering individual data pipelines, enabling teams to onboard sources, protect sensitive data, and promote work to production through configuration instead of custom code. The role is based in San Francisco under a hybrid model. The team works in the office Tuesdays, Wednesdays, and Thursdays, and remote placement is not considered. The position primarily follows local Pacific Time hours, with flexibility across time zones when needed, and travel to Happen Bank offices or other locations as needed. Relocation may be offered based on job level. The target base salary range is $197,000–$230,000. The role may also be eligible for long-term equity awards and an annual bonus. Happen Bank offers medical, dental, and vision plans for employees and their families, a 401(k) match, health and wellness programs, flexible time-off policies for salaried employees, and up to 16 weeks of paid parental leave.

Requirements

10+ years of software or data platform engineering experience, including designing and delivering platform-level frameworks, libraries, or infrastructure used across multiple engineering teams.
Bachelor's degree or higher, or an equivalent combination of education and experience.
Strong infrastructure-as-code experience, including Terraform and CI/CD pipelines for data platforms.
Deep experience building ingestion capabilities across diverse sources, including relational databases, files, APIs, and custom systems, with expertise in incremental processing, change data capture (CDC), schema evolution, and protecting production source systems from excessive load.
Advanced programming skills in Python, SQL, and a JVM language such as Scala or Java, including experience developing Spark UDFs and reusable libraries with strong testing, packaging, and versioning practices.
Deep hands-on expertise with Databricks, including Delta Lake, Unity Catalog, Auto Loader, workflows, compute configuration, and governance capabilities, or equivalent data platform experience.
Strong knowledge of security and privacy engineering for sensitive data, including encryption, key management, tokenization or format-preserving techniques, role-based access controls, and audit logging.
Experience designing and operating large-scale data systems processing billions of rows, including performance testing, cost optimization, monitoring, and recovery strategies.
Excellent written and verbal communication skills, including developing technical designs, architecture decision records, interface contracts, and documentation, and influencing senior stakeholders without direct authority.
Ability to navigate ambiguous technical challenges, evaluate tradeoffs thoughtfully, and own delivery of durable solutions that improve engineering effectiveness across teams.
Deep experience applying AI tools to complex engineering work, including architecture exploration, development, testing, and problem-solving, with sound judgment about security, accuracy, maintainability, and human oversight.
Ability to work primarily local Pacific Time hours and flex across time zones when necessary.
Ability to attend the San Francisco office Tuesdays, Wednesdays, and Thursdays; remote placement is not considered.
Ability to travel to Happen Bank offices and/or other locations as needed.

Nice To Haves

Experience in banking, lending, or another regulated environment with model risk management, SOX, or similar audit requirements.
Experience with data quality, lineage, catalog tooling, and data contract approaches.
Prior experience as a Principal Engineer, Staff Engineer, or technical lead for a platform engineering team.
Experience with Databricks Asset Bundles or equivalent deployment tooling.

Responsibilities

Own the technical vision and architecture for a declarative, configuration-driven ingestion framework that brings relational database and object-store file data into the raw data layer; establish reusable patterns and contracts for downstream teams and evaluate ingestion tradeoffs.
Design and implement reusable, versioned libraries for policy-driven data protection, including configurable rules, governed access, consistent outputs, secure-by-default failure behavior, and audit trails; maintain backward compatibility across shared capabilities.
Define infrastructure-as-code standards for safe deployment, replacement, rollback, and lifecycle management, including data ownership for managed pipeline objects.
Establish compute, cost, and sizing guidance for large initial loads and steady-state incremental processing at billion-row scale.
Establish monitoring, alerting, reconciliation, recovery, and data validation capabilities for reliable source-to-destination operations, and address observability gaps in platform tools.
Conduct structured proofs of concept with measurable success criteria, document architectural decisions and tradeoffs, and work with technology vendors to resolve product gaps and influence future capabilities.
Translate ambiguous technical challenges into clear requirements, interfaces, dependencies, and phased delivery plans.
Partner with Data Engineering, Data Governance, Information Security, DBAs, and source-system owners to deliver early solutions, build scalable long-term capabilities, and drive framework adoption.
Mentor senior and staff engineers, lead technical design and code reviews, and establish shared-platform best practices for testing, documentation, release management, and maintainability.
Help shape the incorporation of AI into engineering workflows while applying judgment about security, accuracy, maintainability, and the need for human oversight.