Senior Data Engineer
About The Position
This hybrid role is based at the Austin Technical Center, where the successful candidate is expected to report at least three times per week, or more often if business needs require. The role focuses on productionizing machine learning for manufacturing and building the data engineering that supports those models. The person will design, train, validate, and deploy models, and design and operate ETL/ELT pipelines, feature data, and governed Bronze, Silver, and Gold data products. Current working knowledge of Databricks is required. The role includes building production model and data pipelines in Databricks, modernizing legacy ETL and proof-of-concept models, and partnering with data science, architecture, product, manufacturing, and platform teams. It also includes operational support and participation in an on-call rotation for production data and ML pipelines.
Requirements
- At least 6 years of experience in machine learning engineering, data engineering, or both.
- Bachelor’s degree in Computer Science, Software Engineering, Data Science, or a related field, or equivalent experience.
- Proven experience deploying machine learning models into production and building supporting data or feature pipelines.
- Strong proficiency in Scala, Python, SQL, Spark, and Databricks, with production-quality software engineering practices.
- Strong experience building and supporting cloud-based ETL/ELT pipelines, data platforms, and orchestration patterns for batch and streaming operations.
- Experience with MLflow or an equivalent model lifecycle, including experiment tracking, registry, versioning, and promotion across environments.
- Strong experience with Databricks jobs, workflows, repos, Unity Catalog, and MLflow-based experimentation and model lifecycle management.
- Experience with CI/CD and GitHub-based deployment of data and ML products.
- Experience delivering production pipelines in collaboration with data science, architecture, product, and business partners.
- Strong troubleshooting, communication, and technical leadership skills.
Nice To Haves
- Data science experience, including building and training models, feature design, model evaluation, and partnering with data scientists to turn experiments into production candidates.
- Experience deploying a model into an isolated network, such as a plant or other restricted environment where runtime is separated from the training platform.
- Experience with Scala, Azure Event Hub, Kafka, or Pulsar.
- Experience with DLT, Auto Loader, structured streaming, and metadata-driven pipeline frameworks.
- Experience with model monitoring, drift detection, validation gates, and production-readiness reviews.
- Experience with manufacturing data, plant-floor systems, industrial telemetry, or manufacturing quality use cases.
- Experience modernizing legacy ETL or moving proof-of-concept work into a governed production pattern.
- Experience working with globally distributed teams and concurrent initiatives.
Responsibilities
- Productionize manufacturing machine learning, including training, validation, batch scoring, deployment, monitoring, and retraining workflows in Databricks.
- Design and build ingestion, transformation, curation, feature pipelines, and governed tables and volumes in Unity Catalog.
- Build reusable Bronze, Silver, and Gold pipelines and medallion patterns for analytics and machine learning.
- Use MLflow and Unity Catalog for experiment tracking, model registration, versioning, lineage, and controlled promotion across environments.
- Ingest enterprise and plant data using managed connectors, DLT, Auto Loader, and structured streaming where appropriate.
- Partner with data science, architecture, product, manufacturing, and platform teams using Python, Spark, SQL, and Databricks-native tools.
- Build CI/CD processes for model and data products, including tests, validation gates, deployment, code review, and operational handoff.
- Improve reliability through data validation, evaluation sets, monitoring, alerting, documentation, and runbooks.
- Support migration from legacy and on-premises ETL and proof-of-concept models into production Databricks patterns.
- Participate in operational support and the on-call rotation for production data and ML pipelines.
Benefits
- Medical, dental, and vision benefits.
- Health Savings Account and Flexible Spending Accounts.
- Retirement savings plan.
- Sickness and accident benefits.
- Life insurance.
- Paid vacation and holidays.
- Tuition assistance programs.
- Employee assistance program.
- GM vehicle discounts.
- Potential eligibility for relocation benefits.
- Incentive pay program with payouts based on company performance, job level, and individual performance.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
