Data Platform Engineer

Rain•New York, DC

1d•$136,384 - $169,738•Hybrid

About The Position

Rain is seeking a Data Platform Engineer to own the data platform used by its data team and engineering pods. The platform supports highly sensitive cardholder, ledger, and partner data across multiple countries, and encompasses the warehouse, batch and streaming ingestion and processing, orchestration, BI, ML capabilities, access controls, governance, and platform tooling. The engineer’s customers are the engineers, analysts, and scientists who build on the platform, and the goal is to make it safe, reliable, compliant, and cost-efficient. The role involves building and operating data platforms for batch and streaming workloads, providing infrastructure and tooling as code, and automating access, deployments, and governance. It also includes supporting ML capabilities for fraud and risk models, advising backend teams on event design, monitoring performance and costs, and responding to incidents and access requests. Rain describes its work environment as open and flexible, with team members encouraged to shape ideas and the company roadmap.

Requirements

At least 4 years building and running production data platforms on a cloud provider such as GCP, AWS, or Azure, covering both batch and streaming infrastructure.
Experience designing data platforms at scale, including storage and compute choices, multi-team access, reliability, and cost.
Experience operating cloud data warehouses and real-time stores such as BigQuery and ClickHouse.
Experience operating streaming infrastructure such as Pub/Sub, Kafka, Dataflow, or Flink, including retention, schema evolution, replay, and lag monitoring.
Experience operating batch infrastructure, including orchestration with Dagster or Airflow, CI/CD, and environments.
Experience with infrastructure as code using Terraform.
Experience with access control and data governance, such as IAM, column- or row-level security, and PII handling.
Strong Python and SQL skills.
Proficiency using AI tools to learn new skills and accelerate work, while taking responsibility for checking their output.

Nice To Haves

ML infrastructure experience, such as feature stores, Vertex AI, or model serving.
Experience in regulated environments such as payments, banking, PCI, or SOC 2.
Experience with data catalogs and lineage tools.
Experience optimizing costs in BigQuery, ClickHouse, or Snowflake.
Experience with CDC tools such as Datastream or Debezium.

Responsibilities

Design and build data platforms at scale for batch and streaming, selecting storage, compute, and processing according to latency, cost, and reliability needs.
Own infrastructure for BigQuery, streaming technologies such as Pub/Sub or Kafka and Dataflow or Flink, Dagster+, and Lightdash as code, including environments, CI/CD, and deployments.
Own access and governance, including IAM, policy tags, PII contracts, masked and no-PII access paths, and audit logs across tables and streams.
Provide ML platform capabilities, including feature pipelines, training data, and serving real-time features to fraud and risk models.
Build platform tooling such as templates, CI checks, schema registry and data contracts, catalog, and lineage.
Monitor platform reliability, latency, and cost, and address slow or expensive components.
Design how backend events and CDC land in the warehouse and advise backend pods on event design.
Respond to platform incidents and access requests, and automate repetitive requests and tasks.

Benefits

Unlimited vacation, with a requirement to take at least 10 days off.
Flexible workplace, with the option to work from home, an office, or both.
A stipend for new employees to set up a home workspace.
For US employees, 95% coverage of employee health, dental, and vision plan costs and 90% coverage for dependents.
Company-subsidized life insurance plan for US employees.
401(k) with a 4% company match.
Equity option plan for every employee.
Monthly stipend for eligible health and wellness spending.
Lunch and dinner covered by DoorDash credit for employees working from the office.
Domestic and international team and company offsites.