Data, ML Ops and Analytics Engineer

OCTAGOS HEALTH, INC.•Houston, TX

17h

About The Position

We are hiring a Data & Analytics Engineer to own the data and intelligence layer that powers clinical reporting, operational analytics, and AI workloads. You will design and build the large-scale Spark and Databricks pipelines that process device and clinical data, own data quality and lifecycle management across the platform, and build the MLOps and feature pipelines behind our AI-assisted workflows. This role sits at the center of engineering, product, clinical operations, and our AI program.

Requirements

Five or more years building data pipelines and analytics solutions in production.
Deep hands-on experience with Apache Spark and Databricks, including PySpark, Delta Lake, notebook-based development, and workflow orchestration.
Strong Python skills for data transformation, automation, and pipeline development.
Strong SQL skills, including T-SQL, query optimization, and dimensional data modeling.
Experience migrating existing T-SQL stored procedures and queries into equivalent Spark and PySpark pipelines.
Experience designing semantic or metrics layers and delivering analytics and dashboards to end users.
Experience with Azure data services such as Azure SQL, Data Factory, Synapse, and Data Lake.
Experience designing data quality frameworks, including validation, monitoring, and remediation.
Experience with data lifecycle management, including retention, archival, governance, and lineage.
Experience with MLOps, including model deployment, versioning, monitoring, drift detection, and feature pipelines.
Clear written communication and the ability to document data assets for technical and non-technical audiences.

Nice To Haves

Experience with healthcare data and working under HIPAA and PHI handling requirements.
Familiarity with MLflow, model registries, feature stores, or model monitoring tools.
Experience with streaming or near-real-time pipelines, such as Spark Structured Streaming.
Familiarity with LLM evaluation, agentic workflows, prompt engineering, or LLM-assisted development.
Experience with DBT or a similar transformation framework.
Familiarity with .NET framework and Dapper.
Relevant cloud or data certifications, such as Databricks, Azure, or AWS data credentials.

Responsibilities

Design, build, and maintain large-scale ETL/ELT pipelines on Spark and Databricks that process device transmission and EMR data.
Develop and tune PySpark jobs and Delta Lake tables for performance, reliability, and cost at scale.
Build dimensional models and a conformed semantic layer that drive consistent metrics across clinical and operational reporting.
Model curated data into the Azure SQL serving layer and develop datasets and embedded dashboards for clinic-facing and internal analytics.
Design and own data quality frameworks, including validation rules, monitoring, anomaly detection, and remediation across the pipeline.
Own data lifecycle management across the platform, including ingestion, retention, archival, lineage, and disposal aligned to HIPAA and compliance requirements.
Build and operate MLOps pipelines for model deployment, versioning, monitoring, and retraining, supporting Atlas AI and other models.
Build feature and measurement pipelines for AI-assisted workflows, establishing ground truth and tracking accuracy, precision, recall, and quality over time.
Monitor model and data drift in production, detecting distribution shifts and performance degradation and triggering retraining or remediation.
Orchestrate, schedule, and monitor data workflows for reliability at scale.
Translate requirements from product, clinical operations, and the AI team into reliable, well-documented data assets.