Senior Data Ops Engineer

Fractal Analytics•New York, NY

6d•$130,000 - $160,000

About The Position

We are hiring a Senior DataOps Engineer to drive enterprise DataOps capabilities, ensuring reliable, scalable, and production-ready data pipelines on Azure Databricks platforms. This role focuses on pipeline orchestration, CI/CD automation, monitoring, and data platform operations to support analytics and AI/ML workloads.

Requirements

Azure Databricks (Mandatory) – PySpark, SQL
Azure Services – ADF, ADLS, Synapse
Data Orchestration – Airflow / ADF / Databricks Workflows
CI/CD Tools – Azure DevOps / GitHub / Git
Strong expertise in ETL/ELT pipelines, DataOps, and production support
Experience with data quality, monitoring, and observability frameworks

Nice To Haves

Experience with Snowflake / BigQuery / multi-cloud environments
Exposure to MLOps / AI data pipelines
Knowledge of Medallion Architecture (Bronze–Silver–Gold)
US Healthcare domain knowledge: Claims, Member, Provider, Eligibility data
Familiarity with FHIR / HL7 / EMR data models

Responsibilities

Build and manage end-to-end data pipelines using Azure Databricks (PySpark, SQL)
Implement workflow orchestration (Airflow / ADF / Databricks Workflows)
Establish Data Ops practices: monitoring, alerting, and production support
Drive CI/CD pipelines and environment promotion (Dev → Prod)
Ensure data quality, lineage, and governance compliance
Manage incident resolution, RCA, and SLA adherence
Optimize pipeline performance, scalability, and cost
Data Ops function ensures consistent, reliable data delivery and operational stability across platforms