Sr Data Engineer

Subway•Shelton, CT

6d•Onsite

About The Position

The Sr Data Engineer is a senior-level, hands-on technical leader responsible for designing, building, and evolving Subway’s enterprise data platform on Snowflake or Databricks. This role serves as a technical authority and builder, driving Lakehouse architecture, engineering frameworks, and best practices across multiple data domains. The Sr Data Engineer operates with a high degree of autonomy, leading through working code and influence — shipping reference implementations, POCs, and platform-level solutions that other teams build upon.

Requirements

Exceptional hands-on expertise in Databricks or Snowflake lakehouse platforms.
Deep proficiency in PySpark or advanced SQL, plus Python for data engineering and automation.
Proven experience building Medallion architecture with Delta Lake or Iceberg Tables.
Real-time and batch streaming experience (Lambda or Kappa) using Databricks DLT or Snowflake Dynamic Tables / Snowpipe Streaming.
Hands-on with orchestration tools — Airflow, Databricks Lakeflow, or Snowflake Openflow; dbt experience a plus.
Strong data modeling skills (Dimensional, Data Vault, schema design).
Performance and cost tuning expertise (clustering, partitioning, Z-ordering, warehouse/cluster sizing, FinOps).
Governance experience with Unity Catalog (Databricks) or Horizon Catalog (Snowflake) for lineage, access control, and data quality.
Semantic layer experience using Databricks AI/BI Genie / Unity Catalog Metrics or Snowflake Semantic Views / Cortex Analyst.
AI/ML enablement experience with Databricks Mosaic AI or Snowflake Cortex.
CI/CD and DevOps fluency — Git, Databricks Asset Bundles or Snowflake CLI / Schemachange, automated testing.
Cloud ecosystem expertise — AWS (S3, Glue, Kinesis), Azure, or GCP.
Excellent communication and technical storytelling ability.
Comfortable operating across ambiguity and complex stakeholder environments.
Bachelor’s degree required (Computer Science, Engineering, or related field); advanced degree preferred.
3-5 years of professional data engineering experience, with proven track record leading architecture and hands-on build for enterprise-scale data platforms, operating in complex or mission-critical data environments, and influencing multiple teams and platforms without direct authority.

Responsibilities

Personally design and build reference implementations and production-grade frameworks on Databricks or Snowflake.
Design lakehouse platforms using Delta Lake or Iceberg Tables with Medallion (Bronze/Silver/Gold) architecture.
Define and evolve enterprise data standards, patterns, and reusable accelerators.
Ensure solutions align with data governance, security, scalability, and cost-efficiency standards.
Evaluate technologies through hands-on benchmarking — not vendor decks.
Build the first working version of complex pipelines, frameworks, and POCs (ingestion, CDC, streaming, DQ, observability, CI/CD).
Drive emerging tech (Iceberg, Lakeflow, Openflow, Cortex, Mosaic AI) from POC to production rollout.
Solve high-complexity performance, cost, and governance challenges at petabyte scale.
Identify and address systemic technical debt and architectural risks.
Implement Lambda or Kappa architectures using Databricks Structured Streaming / DLT or Snowflake Dynamic Tables / Snowpipe Streaming.
Build GenAI and ML enablement patterns (RAG, feature stores, semantic layers) using Databricks Mosaic AI or Snowflake Cortex.
Partner with Data Science and Analytics teams to operationalize models and AI workflows.
Collaborate closely with Product, Architecture, Security, Infrastructure, and Analytics leaders.
Translate business needs into sound technical direction backed by working prototypes.
Communicate technical trade-offs, risks, and decisions clearly to technical and non-technical stakeholders.
Influence roadmaps and platform investments through technical insight and de-risking POCs.
Mentor Senior and Staff Data Engineers through pair-programming, PR reviews, and design coaching.
Raise engineering maturity by shipping working examples and codifying patterns.
Foster a culture of technical excellence, learning, and continuous improvement.