Principal Software Engineer, Data Platform — Data Trust & AI Context
About The Position
As a Principal Software Engineer in SoFi’s Data Platform Group, you will provide hands-on technical leadership within the Data Trust pillar. You will design, build, and operate distributed systems and reusable platform primitives that make data quality, privacy, metadata, lineage, access, and reliability enforceable by construction across the enterprise. The role spans shared Data Trust capabilities, improving reliability across batch and low-latency environments, and establishing a semantic context layer for trusted analytics and AI. You will make business meaning, ownership, provenance, quality, freshness, and access policies machine-readable so end users and AI systems can use consistent, governed context. This is an individual contributor role with broad organizational impact: you will contribute code, prototypes, architecture, and operational improvements while setting technical direction across engineering, data, security, privacy, legal, risk, analytics, and product teams. You will build reusable enterprise capabilities and reference patterns for partner teams, while domain teams retain ownership of application-specific use cases and serving paths. You will define and execute Data Trust architecture and strategy for data contracts, metadata and catalog modernization, lineage, data quality, privacy, and access controls. You will build platform services, libraries, and reference patterns that make requirements versioned, testable, observable, and automatically enforceable across the data lifecycle. You will establish foundations for a governed semantic context layer and evolve Data Trust capabilities across Snowflake, AWS data services, Kafka-based ingestion, and other batch or low-latency environments. The role also focuses on production reliability and operational rigor, including explicit SLOs, failure modes, correction paths, and ownership, supported by automated detection, containment, remediation, and recovery. You will lead multi-quarter initiatives across teams, align stakeholders on architectural tradeoffs and investments, mentor senior engineers, and strengthen design and implementation practices. In the first 12–18 months, success includes establishing an enterprise architecture and delivery path for Data Trust primitives; materially improving coverage, observability, SLOs, and automated enforcement for priority data domains; delivering production-ready governance patterns for batch and low-latency data; operating initial semantic context capabilities in production; reducing manual investigation, recurring failures, and compliance evidence-gathering; and driving adoption of reusable standards and capabilities.
Requirements
- 15+ years of software or data engineering experience, or 13+ years with a Master’s degree.
- Proven track record designing, building, and operating distributed data platforms at enterprise scale.
- Deep experience designing horizontal, multi-tenant platform services used by other engineering teams.
- Understanding of distributed-system failure modes, eventual consistency, and tradeoffs among latency, availability, and cost.
- Practical experience building or managing data trust stack components, such as data contracts, metadata catalogs, data quality frameworks, end-to-end lineage, and fine-grained access control (RBAC/ABAC).
- Advanced knowledge of modern data ecosystems, specifically Snowflake, Kafka or similar streaming platforms, and AWS data services.
- Ability to govern data in motion as effectively as data at rest.
- Mastery of one or more core programming languages, such as Python, Java, or Go, and advanced SQL.
- Comfort writing production-grade code, conducting rigorous code reviews, and building declarative infrastructure using tools such as Terraform and CI/CD.
- History of driving multi-team, cross-functional technical initiatives and building consensus across Engineering, Security, Privacy, and Product teams.
- Ability to translate compliance and risk requirements into engineering primitives.
- Experience managing Tier-0 or mission-critical infrastructure.
- Trust-but-verify operational approach, including automated testing, drift detection, and circuit breakers rather than manual processes.
Nice To Haves
- Experience operating in a highly regulated environment where auditability, data privacy (CCPA/GDPR), and data integrity are architectural requirements.
- Experience building semantic layers, knowledge graphs, or context engines that ground LLMs and autonomous agents in governed enterprise data.
- Contributions to relevant open-source data or metadata projects, such as Apache Iceberg, OpenMetadata, DataHub, or Amundsen.
Responsibilities
- Define and execute technical direction for data contracts, metadata and catalog modernization, lineage, data quality, privacy, and access controls.
- Translate business, regulatory, and engineering requirements into a coherent platform strategy and sequence investments for long-term impact.
- Build platform services, libraries, and reference patterns that make contracts, quality rules, privacy policies, and access requirements versioned, testable, observable, and automatically enforceable.
- Establish technical foundations for a governed semantic context layer connecting business definitions, metadata, lineage, ownership, quality, freshness, and access policy.
- Enable trusted discovery, analytics, and AI-assisted or agentic experiences without making the role a model-development function.
- Lead the evolution of Data Trust capabilities across Snowflake, AWS data services, Kafka-based ingestion, and other batch or low-latency environments.
- Establish explicit SLOs, failure modes, correction paths, and operational ownership, supported by automated detection, containment, remediation, and recovery.
- Improve production data systems through observability, integrity checks, audit evidence, incident learning, and safe automation.
- Drive multi-quarter initiatives across teams and functions and build alignment on architectural tradeoffs and investment decisions.
- Serve as a technical authority for Data Trust, mentor senior engineers, strengthen design and implementation practices, and raise engineering standards.
- Contribute hands-on code, prototypes, architecture, and operational improvements.
- Develop reusable enterprise capabilities and reference patterns that partner teams can adopt.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Principal
