Senior Data Engineer - Databricks (No Sponsorship Available)

Heritage Construction + Materials • Indianapolis, IN

2d

About The Position

The Senior Data Engineer will help develop and implement strategic data initiatives. They will be responsible for developing and maintaining data pipelines, promoting sustainable data governance, and serving as the organization’s cloud data platform expert.

Requirements

• 5+ years of experience in data engineering or a related field
• Expertise with programming languages such as Python/PySpark, SQL, or Scala
• Experience working in a cloud environment (Azure preferred) with strong understanding of cloud data architecture
• Hands-on experience with Databricks or Snowflake Cloud Data Platforms
• Experience with workflow orchestration (e.g., Databricks Jobs, or Azure Data Factory pipelines)

Nice To Haves

• Knowledge of statistical analysis and data visualization tools, such as PowerBI or Tableau
• Understanding of business financials and NPV analysis
• Creative thinker that is open to working collaboratively with a team

Responsibilities

• Develop and support data pipelines within a Cloud Data Platform, such as Databricks or Snowflake
• Monitor and optimize Databricks cluster performance, ensuring cost-effective scaling and resource utilization
• Implement and maintain Delta Lake for optimized data storage, ensuring data reliability, performance, and versioning
• Automate CI/CD pipelines for data workflows using Azure DevOps
• Integrate Databricks with other Azure services, such as Azure Data Factory, Azure Synapse, and ADLS
• Design and implement monitoring solutions, leveraging Databricks monitoring tools, Azure Log Analytics, or custom dashboards for cluster and job performance
• Collaborate with cross-functional teams to support data governance through the use of Databricks Unity Catalog
• Additional duties and responsibilities as assigned, including but not limited to continuously growing in alignment with the Company’s core values, competencies, and skills.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior