Staff Engineer - Data Engineering

Early Warning® • San Francisco, CA

1d • $124,000 - $223,000 • Hybrid

About The Position

This position provides senior technical leadership and hands on work in the architecture, design, development, testing, and deployment of complex data platforms and data-driven solutions. The role partners across engineering, product, and data teams to define technical strategy, solve large-scale and open-ended problems, and deliver scalable, reliable solutions from concept through production. This is a hands-on role where 50+% of time will be spent coding and design.

Requirements

• Bachelor’s degree in computer science or a related technical field.
• Eight or more years of relevant professional experience.
• Seven or more years of experience developing complex data platforms, distributed systems, SaaS solutions, cloud-based solutions, and/or microservices.
• Six or more years of experience developing data warehouse, data lake, lakehouse, and big data platforms, including structured and unstructured data, real-time and batch processing, and data standards.
• Two or more years of experience developing and/or operationalizing Artificial Intelligence or Machine Learning models, including feature engineering, data pipelines, model deployment, operationalization, and monitoring.
• Demonstrated experience delivering business-critical systems to production.
• Experience designing and developing scalable, highly available systems.
• Extensive experience implementing modern data warehouse solutions using technologies such as Snowflake, including dimensional modeling, star and snowflake schemas, performance optimization, and scalable data processing.
• Experience designing and implementing modern data lake and lakehouse architectures using Databricks, Apache Iceberg, Amazon S3 or equivalent technologies.
• Experience developing scalable ETL/ELT, data integration, and reusable data processing frameworks.
• Experience implementing data engineering, data science, or analytics solutions using Python, PySpark, Apache Spark, Databricks, Snowflake, or equivalent tools.
• Experience with event-driven architecture and messaging frameworks such as Amazon SNS, Amazon SQS, Amazon Kinesis, Apache Kafka, or equivalent technologies.
• Working experience with AWS cloud infrastructure and services, including scalable compute, storage, networking, security, and data services.
• Experience with Infrastructure as Code (IaC) and AWS infrastructure provisioning using Terraform or equivalent technologies.
• Knowledge of mature engineering practices, including CI/CD, automated testing, secure coding, code quality standards, and automated infrastructure deployment.
• Knowledge of Software Development Lifecycle best practices, software development methodologies such as Agile, Scrum, and Lean, and DevOps practices.
• Strong attention to detail.
• Background and drug screen.

Nice To Haves

• Master’s or doctoral degree in computer science, data science, engineering, or a related technical field.
• Deep experience designing and operating large-scale AWS-based data platforms.
• Strong hands-on experience with Databricks, including scalable data processing, data pipelines, workload optimization, and lakehouse architecture.
• Strong hands-on experience with Snowflake, including data modeling, ingestion, performance optimization, workload management, and enterprise-scale data warehouse architecture.
• Experience integrating Databricks and Snowflake within modern enterprise data architectures.
• Experience with modern open table formats and lakehouse technologies such as Apache Iceberg and Delta Lake.
• Experience designing and operating cloud-based data lake and object-storage solutions using Amazon S3.
• Strong programming experience with Python, PySpark, and Apache Spark.
• Experience implementing AWS infrastructure using Terraform and Infrastructure as Code practices.
• Experience with AWS data and analytics services such as Amazon S3, AWS Glue, Amazon Airflow , Amazon Kinesis, Amazon Redshift, AWS Lambda, and Amazon Athena.
• Experience with data governance, cataloging, lineage, metadata management, and data quality capabilities across Databricks, Snowflake, and AWS.
• Experience with monitoring, alerting, logging, and observability for cloud-based data platforms.
• Kubernetes and container orchestration experience, including Amazon EKS.
• Experience designing and operating event-driven, streaming, and real-time data architectures using AWS services and technologies such as Kinesis and Kafka.
• FinTech or payments industry experience.

Responsibilities

• Drive technical and data architecture, design, prototyping, and implementation in support of product needs and the broader technology and data strategy.
• Provide technical leadership in developing technical plans and backlogs and drive execution from design and development through production delivery.
• Own features, platforms, or systems and define their long-term health, reliability, scalability, and maintainability while improving surrounding systems.
• Partner with Support and Operations teams to identify, troubleshoot, and quickly resolve production issues.
• Develop and implement testing strategies to ensure application quality, performance, reliability, and scalability.
• Identify and drive improvements to engineering and data standards, tooling, development practices, and processes.
• Support the company’s commitment to risk management and to protecting the integrity, availability, and confidentiality of systems and data.

Benefits

• Competitive medical (PPO/HDHP), dental, and vision plans as well as company contributions to your Health Savings Account (HSA) or pre-tax savings through flexible spending accounts (FSA) for commuting, health & dependent care expenses.
• 100% Company Safe Harbor Match on your first 6% deferral immediately upon eligibility.
• Flexible Time Off for Exempt (salaried) employees, as well as generous PTO for Non-Exempt (hourly) employees, plus 11 paid company holidays and a paid volunteer day.
• 12 weeks of Paid Parental Leave
• Maven Family Planning – provides support through your Parenting journey including egg freezing, fertility, adoption, surrogacy, pregnancy, postpartum, early pediatrics, and returning to work.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior