Senior Data Engineer (Hybrid, Houston based)

Balco, Inc.•Houston, TX

2d•Hybrid

About The Position

We are seeking an experienced Senior Data Engineer to join our Data & BI team within the manufacturing and distribution sector. This role will focus on implementing and maintaining an Azure cloud-based data platform and building scalable medallion architecture solutions to support enterprise data initiatives. This is a delivery-focused, hands-on role with primary responsibilities in implementation, platform maintenance, operational excellence, and supporting modernization initiatives. This hybrid role will require a presence at our Houston Spenwick facility for two to three days during regular business hours.

Requirements

Databricks: 5+ years of hands-on data engineering experience. Strong Python skills. Proficient with Databricks — Delta Lake, PySpark/Spark SQL, notebooks, jobs, and cluster management
Azure Data Platform: Proficient with Azure, Azure Data Lake Storage Gen2, and the broader Azure data ecosystem, Azure Data Factory, Azure Logic Apps, and Azure Key Vault
Data Integration: Practical experience with connector management, API, and Databricks Lakeflow (Workflows / DLT pipelines). Hands-on experience with medallion architecture, dimensional modeling, and incremental load
Source Systems: 5+ years working with Oracle (PL/SQL, stored procedures) and SQL Server (T-SQL, CDC, query optimization); experience extracting from JD Edwards (JDE) table structures and business data models
Access Control: Hands-on experience implementing RBAC/ABAC, row-level security, and column-level security across Databricks, Unity Catalog, and relational databases
DevOps: Experienced with Git, Azure DevOps or GitHub Actions, ansible and CI/CD pipelines
Knowledge and hands-on experience with Power BI
Familiar with Agile methodology, participate in Sprint planning
Excellent communication skills, with the ability to translate complex data into actionable business insights

Nice To Haves

Exposure to streaming data patterns (Event Hubs, Kafka, or Structured Streaming)
Familiarity with Azure Functions, Azure ML, Logic Apps, Azure Synapse, Microsoft Fabric, and Oracle Cloud
Knowledge of FiveTran, dbt, Machine Learning, and AI development and deployment

Responsibilities

Implement cloud-based scalable data platforms to support manufacturing operations, inventory management, and distribution analytics.
Build and maintain Lakehouse and develop ELT/ETL workflows using Databricks, Lakeflow, Azure Data Factory, FiveTran, and Azure Data Lake Storage
Integrate data from ERP and different source systems, including JD Edwards (JDE), Hubspot, Oracle, SQL Server, etc.
Configure and maintain Unity Catalog — applying access policies, lineage settings, and metadata as directed by the architectural governance framework.
Implement RBAC/ABAC across the data platform, including row-level and column-level security in UC and ADLS
Monitor pipeline health, troubleshoot failures, and perform root-cause analysis on data quality or performance issues
Participate in daily support and respond to production incidents in a timely manner
Actively share knowledge and produce clear technical documentation for all implemented solutions — including pipeline designs, data flow diagrams, runbooks, and configuration notes
Maintain up-to-date documentation in the team's knowledge base
Contribute to team standards and best practices