Sr. Data Engineer

Prolaio • Chicago, IL

7d

About The Position

As a Senior Data Engineer, you will design, build, and operate data pipelines in a regulated environment. You will work closely with data science and clinical teams, which own the medical and research questions, and collaborate with the Data Architect on platform standards, data models, and governance.

You will build and evolve Prolaio’s data ecosystem, including batch, streaming, and hybrid pipelines orchestrated with Dagster; dbt models in BigQuery; and ingestion and normalization of structured and unstructured sources such as cardiology wearables and device signals, EHR, claims, medication, and patient-reported data. You will make decisions about storage, compute, partitioning, cost, and APIs, and contribute to target-state architecture and reference patterns.

The role includes implementing data lifecycle controls and traceability, supporting validated operation in a regulated setting, and backing pipeline changes with automated tests and data quality checks. You will enable data science and ML workflows, contribute to internal applications and Python libraries, own data platform CI/CD using GitLab CI, help monitor pipeline health, performance, and BigQuery cost with Datadog, participate in incident response, and help mentor engineers and establish team standards.

Requirements

• At least 6 years of experience in data engineering.
• Expertise in Python and SQL.
• Experience with GCP or AWS/Azure.
• Production experience with dbt, including modeling, testing, macros, and incremental models.
• Experience with BigQuery or a comparable cloud data warehouse such as Databricks or Redshift.
• Experience with Dagster or a comparable orchestrator such as Airflow or Prefect.
• Experience with Docker and large-scale data processing.
• Experience with structured, semi-structured, and unstructured data, including time-series data.
• Understanding of MLOps concepts and production ML workflows.
• Experience collaborating with data scientists, product teams, and clinical or quality stakeholders.
• Experience with Kubernetes.

Nice To Haves

• Hands-on GCP experience, including IAM, Cloud Storage, Cloud Run, Pub/Sub, and networking.
• Infrastructure as code experience with Terraform.
• Experience with Datadog.
• Experience with metadata, lineage, and catalog tools, plus retrieval or vector tooling for AI use cases.
• Experience with Databricks.
• Experience working with healthcare, life sciences, medtech, or other patient data.
• Experience in regulated or validated environments such as SaMD, FDA, GxP, or HIPAA.
• Experience with streaming tools such as Pub/Sub or Kafka.

Responsibilities

• Design, build, and maintain batch, streaming, and hybrid data pipelines orchestrated with Dagster for ingestion, validation, transformation, enrichment, and publishing.
• Develop and maintain dbt models in BigQuery across staging, intermediate, and mart layers, with tests, documentation, and clear ownership.
• Ingest and normalize heterogeneous structured and unstructured sources, including cardiology wearables and device signals, EHR, claims, medication, and patient-reported data.
• Make decisions about storage, compute, partitioning, cost, and APIs, and contribute to target-state architecture and reference patterns.
• Contribute data lifecycle controls for retention, access control, lineage, provenance, reproducibility, and auditability.
• Use Dagster assets and dbt lineage to trace data from source to output.
• Document pipelines and maintain traceability while working with Quality and Security teams to support validated operation in a regulated setting.
• Back pipeline changes with automated tests and data quality checks.
• Enable data scientists to experiment and iterate efficiently by providing tools and processes.
• Contribute to ML workflows supporting feature generation, training, deployment, and monitoring.
• Contribute to internal applications and Python libraries.
• Own CI/CD for the data platform using GitLab CI, including dbt and Dagster deployments.
• Help monitor pipeline health, performance, and BigQuery cost with Datadog, and participate in incident response.
• Help mentor other engineers and establish engineering standards for the data team.

Benefits

• Competitive salary, performance bonus, and equity.
• Medical, dental, and vision plans with multiple options and strong company contributions.
• HSA, FSA, commuter benefits, and a $1,200 annual Lifestyle Spending Account.
• Generous paid time off, sick leave, and company holidays.
• Paid parental leave, caregiver leave, and support for growing families.
• Company-paid life insurance and short- and long-term disability coverage.
• 401(k) plan.
• Telehealth access and optional supplemental coverage.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed