Senior Data Engineer (Hybrid, Houston based)

Balco, Inc.•Houston, TX

107d•Hybrid

About The Position

The Senior Data Engineer will join the Data & BI team in the manufacturing and distribution sector. The role focuses on implementing and maintaining an Azure cloud-based data platform and building scalable medallion-architecture solutions to support enterprise data initiatives. It is a delivery-focused, hands-on position emphasizing implementation, platform maintenance, operational excellence, and modernization initiatives. The hybrid role requires presence at the Houston Spenwick facility two to three days during regular business hours. The engineer will build and maintain a Lakehouse and ELT/ETL workflows, integrate data from ERP and other source systems, configure data governance and access controls, and monitor and support production pipelines. The role also involves sharing knowledge, maintaining clear technical documentation, and contributing to team standards and best practices.

Requirements

At least 5 years of hands-on data engineering experience with Databricks.
Strong Python skills.
Proficiency with Databricks, including Delta Lake, PySpark/Spark SQL, notebooks, jobs, and cluster management.
Proficiency with Azure, Azure Data Lake Storage Gen2, and the broader Azure data ecosystem, including Azure Data Factory, Azure Logic Apps, and Azure Key Vault.
Practical experience with connector management, APIs, and Databricks Lakeflow (Workflows / DLT pipelines).
Hands-on experience with medallion architecture, dimensional modeling, and incremental loads.
At least 5 years working with Oracle, including PL/SQL and stored procedures, and SQL Server, including T-SQL, CDC, and query optimization.
Experience extracting data from JD Edwards (JDE) table structures and business data models.
Hands-on experience implementing RBAC/ABAC and row-level and column-level security across Databricks, Unity Catalog, and relational databases.
Experience with Git, Azure DevOps or GitHub Actions, Ansible, and CI/CD pipelines.
Knowledge and hands-on experience with Power BI.
Familiarity with Agile methodology and participation in sprint planning.
Excellent communication skills and the ability to translate complex data into actionable business insights.

Nice To Haves

Exposure to streaming data patterns, such as Event Hubs, Kafka, or Structured Streaming.
Familiarity with Azure Functions, Azure ML, Logic Apps, Azure Synapse, Microsoft Fabric, and Oracle Cloud.
Knowledge of FiveTran, dbt, machine learning, and AI development and deployment.

Responsibilities

Implement scalable cloud-based data platforms supporting manufacturing operations, inventory management, and distribution analytics.
Build and maintain a Lakehouse and develop ELT/ETL workflows using Databricks, Lakeflow, Azure Data Factory, FiveTran, and Azure Data Lake Storage.
Integrate data from ERP and other source systems, including JD Edwards (JDE), HubSpot, Oracle, and SQL Server.
Configure and maintain Unity Catalog, applying access policies, lineage settings, and metadata according to the architectural governance framework.
Implement RBAC/ABAC across the data platform, including row-level and column-level security in Unity Catalog and ADLS.
Monitor pipeline health, troubleshoot failures, and perform root-cause analysis of data quality or performance issues.
Participate in daily support and respond to production incidents in a timely manner.
Share knowledge and create technical documentation for implemented solutions, including pipeline designs, data flow diagrams, runbooks, and configuration notes.
Maintain current documentation in the team knowledge base.
Contribute to team standards and best practices.
Participate in sprint planning.

Benefits

Comprehensive medical plans
Dental plans
Vision plans
401(k)
Pension
Life insurance
Paid time off program
Paid holidays
Wellness programs
Employees may be eligible for pay incentives based on overall corporate and individual performance or at the discretion of the CSWI Board of Directors.