Senior Data Engineer

GM•Austin, TX

11d•Hybrid

About The Position

This role is categorized as hybrid. This means the successful candidate is expected to report to Austin Technical Center three times per week, at minimum [or other frequency dictated by the business if more than 3 days]. What You’ll Do Design and develop batch and streaming ETL/ELT pipelines in Azure Databricks to support ingestion, transformation, curation, and delivery of manufacturing and enterprise data. Build reusable Bronze, Silver, and Gold data pipelines, data quality frameworks, and governed data products in Unity Catalog. Build and operationalize ML workflows for data preparation, feature engineering, model training, evaluation, deployment, batch scoring, retraining, and inference. Establish and maintain MLflow-based model lifecycle controls, including experiment tracking, model registration, versioning, lineage, traceability, and controlled promotion across environments. Implement CI/CD pipelines for data products, ML code, workflows, and deployment bundles using Git-based development practices, automated testing, and release automation. Build and support Databricks jobs and workflows for orchestration, scheduled execution, retraining, scoring, and downstream operational consumption. Improve reliability, observability, and production readiness through testing, monitoring, validation, alerting, documentation, and automation that reduce deployment risk and manual intervention. Partner with data scientists, data engineers, platform teams, architecture, manufacturing, and business stakeholders to productionize scalable, supportable solutions aligned to production constraints. Support modernization of legacy data and ML workflows into standardized cloud-native Databricks patterns and help bridge proof-of-concept work into repeatable production solutions.

Requirements

5+ years of relevant full-time experience; or equivalent knowledge
Bachelor’s degree in Computer Science, Data Science, Engineering, or a related technical field, or equivalent practical experience.
Strong proficiency in Scala, Python, SQL, Spark, and Databricks, with production-quality software engineering practices.
Strong experience building and supporting cloud-based ETL/ELT pipelines, data platforms, and pipeline orchestration patterns for both batch and streaming operations.
Experience deploying machine learning models into production environments and supporting model development through operational release.
Strong experience with Databricks jobs, workflows, repos, Unity Catalog, and MLflow-based experimentation and model lifecycle management.
Experience building CI/CD pipelines with Git-based workflows, automated testing, and deployment of data or ML products.
Strong troubleshooting, communication, and cross-functional collaboration skills.

Nice To Haves

Experience with DLT, structured streaming, Event Hub, Kafka, Scala, or Java in enterprise data pipeline environments.
Experience with model governance, lineage, reproducibility, monitoring, validation checks, and operational observability.
Experience designing resilient pipelines that can handle changing upstream data conditions, retraining needs, and production operations.
Experience with manufacturing, industrial IoT, plant-floor data, quality, or manufacturing analytics use cases.
Experience helping transition PoC or R&D solutions into scalable production-ready patterns.

Responsibilities

Design and develop batch and streaming ETL/ELT pipelines in Azure Databricks to support ingestion, transformation, curation, and delivery of manufacturing and enterprise data.
Build reusable Bronze, Silver, and Gold data pipelines, data quality frameworks, and governed data products in Unity Catalog.
Build and operationalize ML workflows for data preparation, feature engineering, model training, evaluation, deployment, batch scoring, retraining, and inference.
Establish and maintain MLflow-based model lifecycle controls, including experiment tracking, model registration, versioning, lineage, traceability, and controlled promotion across environments.
Implement CI/CD pipelines for data products, ML code, workflows, and deployment bundles using Git-based development practices, automated testing, and release automation.
Build and support Databricks jobs and workflows for orchestration, scheduled execution, retraining, scoring, and downstream operational consumption.
Improve reliability, observability, and production readiness through testing, monitoring, validation, alerting, documentation, and automation that reduce deployment risk and manual intervention.
Partner with data scientists, data engineers, platform teams, architecture, manufacturing, and business stakeholders to productionize scalable, supportable solutions aligned to production constraints.
Support modernization of legacy data and ML workflows into standardized cloud-native Databricks patterns and help bridge proof-of-concept work into repeatable production solutions.