Senior Data Platform Architect (Snowflake, Databricks, Open Platforms & Data Virtualization), VP I - State Street Investment Management

State Street • Boston, MA

9d • $120,000 - $202,500

About The Position

The Senior Data Platform Architect is responsible for end-to-end architecture, security design, and governance of State Street Investment Management’s enterprise data platform, spanning Snowflake, Databricks, open data platforms and formats, and data virtualization. The role supports analytics, regulatory reporting, AI/ML, and batch workloads, ensuring the platform meets financial-services regulatory expectations, enterprise security standards, and NPPI/PII protection requirements. It is a hands-on, decision-critical role with accountability for architecture affecting regulatory compliance, audit defensibility, and business continuity. The architect designs secure, scalable platforms with disaster recovery and data-sharing capabilities, and builds monitoring and chargeback models for application teams.

The role defines and governs hybrid Snowflake–Databricks architecture, establishes reusable enterprise patterns, and leads interoperability across cloud environments and proprietary and open data ecosystems. It also architects virtualized and federated data access, modernizes batch applications and legacy workloads, and establishes standards for performance, cost, resiliency, security, governance, and regulatory alignment.

The architect partners with security, risk, audit, application, and cloud engineering teams; serves as a subject-matter expert during architecture reviews and escalations; and produces architecture diagrams, design artifacts, executive presentations, and access models. The position also advises application teams on modernization and security-related upgrades, supports legacy infrastructure retirement and cloud migration, and captures user feedback for vendors.

Requirements

• Experience supporting analytics, regulatory reporting, AI/ML, and batch workloads.
• Snowflake expertise including warehouse sizing, object hierarchy, Snowsight, RBAC, Horizon Catalog, governance, performance, data sharing, external tables, stages, loading, stored procedures, scripts, and Snowpark.
• Databricks expertise including Spark, Unity Catalog, ML pipelines, Delta/Iceberg, and Lakehouse Federation.
• Knowledge of Apache Iceberg, Delta Lake, Parquet, Polaris, and Trino/Starburst.
• Experience with data virtualization, including Denodo, Starburst, Snowflake/Databricks federation, and logical data fabrics.
• Experience modernizing batch workloads from Oracle/PL-SQL, mainframe, Unix ETL, and Control-M to cloud-native orchestration.
• Multi-cloud knowledge of AWS, Azure, and GCP storage, IAM, networking, and private-link capabilities.
• Data modeling knowledge including dimensional and denormalized modeling, medallion schemas, bridge tables, marts, primary and foreign keys, clustering, and tuning implications.
• Programming and data-platform language skills in SQL, Python, Java, JavaScript, Scala, Spark, notebooks, and Snowpark.
• Linux proficiency and comfort working with EC2 servers and command-line tools such as vi, cat, and more.
• Regulated-industry experience, including financial services, SOX, MRA, audit, and NPPI/PII handling.
• Comfort with generative AI scripting and prompting, including knowing when scripting is more efficient and generating scripts with well-defined prompts and skill files.
• Strong teamwork and communication skills, willingness to share knowledge and learn from others, and ability to coordinate with team members, project managers, and other teams.

Nice To Haves

• Previous Redshift, DB2, and SQL Server experience to support migrations and data-conversion issues.

Responsibilities

• Define and govern hybrid Snowflake–Databricks architecture for data warehousing and data lake use cases.
• Establish scalable, auditable reference blueprints and reusable lakehouse and medallion architecture patterns; contribute designs, patterns, and assets to the SSIM Center of Excellence.
• Lead interoperability for Iceberg, Horizon, and Unity Catalog across AWS, Azure, and GCP.
• Architect open data platform strategy using Apache Iceberg, Delta Lake, Parquet, and Polaris/Unity Catalog to support cross-engine queries and reduce vendor lock-in.
• Set open-format standards for access protocols, drivers, naming, folder structures, cloud storage security, schema evolution, partitioning, time travel, engine compatibility, and catalog federation.
• Establish governance for open formats in regulated workloads, including lineage, access control, and encryption, and support modernization and exit strategies across proprietary and open ecosystems.
• Gather user feedback and convey distilled findings and suggestions to vendors to improve their offerings.
• Architect data virtualization and federated access across Snowflake, Databricks, Oracle, SQL Server, and legacy NAS/file systems without unnecessary physical data movement.
• Define use cases for federated queries, virtual data products, and logical data fabrics across cloud and on-premises sources.
• Set performance, cost, security, and data-classification guardrails for virtualized access, reducing data duplication and storage costs where appropriate.
• Lead architecture and solutioning for batch applications migrating from Oracle, mainframe, Unix/Linux ETL, and file-based workflows to Snowflake and Databricks.
• Define batch ingestion, orchestration, SLA-driven processing, monitoring, rerun-safety, legacy code migration, and hybrid batch/streaming patterns.
• Establish batch performance, cost, and resiliency standards, including reprocessing, rollback, event logging, query timeouts, warehouse credit limits, and identification of expensive processes through account-usage tables.
• Partner with application owners to retire legacy infrastructure as part of data-center exit and cloud migration programs.
• Identify outdated drivers and application versions that may pose security risks and advise application teams on upgrades.
• Design data security controls, including group-level RBAC roles, row-access policies, column masking, secure views, future grants, managed-access schemas, network allowlisting, and cloud security integrations.
• Ensure compliance with SOX, MRAs, audits, and data-protection controls; maintain audit trails, lineage, and forensic traceability.
• Align platform design with GCS data-residency, isolation, and classification standards.
• Architect cloud-native integrations across S3, ADLS, GCS, IAM, networking, and private-link patterns.
• Design cross-region and cross-cloud high availability and disaster recovery, including Snowgrid replication and Iceberg federation.
• Embed vendor security, real-time/streaming networking, and disaster recovery in platform designs.
• Serve as a subject-matter expert for Snowflake, Databricks, open platforms, virtualization, and batch modernization during architecture reviews and escalations.
• Partner with Security (GCS/GTS), Risk, Audit, Application, and Cloud Engineering teams.
• Produce architecture diagrams, design artifacts, executive-level presentations, and access models.

Benefits

• Retirement savings plan (401(k)) with company match.
• Basic life, medical, dental, vision, long-term disability, and optional additional insurance coverage.
• Paid time off, including vacation, sick leave, short-term disability, and family-care responsibilities.
• Access to the Employee Assistance Program.
• Eligibility for annual performance-based incentive awards, excluding certain sales roles subject to sales incentive plans.
• Eligibility for certain tax-advantaged savings plans.
• Inclusive development opportunities, flexible work-life support, paid volunteer days, and employee networks.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed