Data Engineer

Sunbelt Rentals, Inc.Fort Mill, SC
20d

About The Position

We are seeking a highly skilled and results-driven Data Engineer to design, develop, and improve strong, scalable data pipelines and systems to support effective data processing, storage, and retrieval. The role must ensure data integrity, security, and accessibility while using modern technologies like cloud computing, big data frameworks, and automation. The position will also work with different teams to provide high-quality, organized data for analytics, machine learning, and business intelligence. All of which supports data-driven decision-making and innovation.

Requirements

  • Bachelor’s degree in computer science, Information Technology, Data Engineering, or equivalent experience.
  • 5+ years of experience in data engineering, database management, and cloud platforms.
  • Proficiency in MS SQL Server, Snowflake, and Databricks.
  • Experience with ETL development, cloud data platforms, and DevOps tools.
  • InDepth understanding of cloud platform like Azure, AWS and GCP.
  • Strong analytical and problem-solving skills, with a focus on end-to-end delivery.
  • Ability to translate business requirements into technical solutions.
  • Excellent communication and leadership abilities.

Nice To Haves

  • Masters degree (strongly preferred)
  • Familiarity with GITC is a plus.

Responsibilities

  • Data Engineering & Architecture - Design and manage data pipelines in Databricks utilizing Apache Spark, PySpark, Delta Lake, Unity Catalog, and Workflows.
  • Establish Medallion architecture (Bronze, Silver, Gold) to develop scalable data lake solutions with dimensional modeling.
  • Create ELT/ETL workflows using Databricks Notebooks and Azure Data Factory.
  • Integrate API-based, structured, and unstructured data across Azure Databricks and hybrid cloud environments.
  • Enhance cluster and workflow performance while optimizing costs.
  • Infrastructure & Automation - Employ Terraform and ARM templates for infrastructure-as-code. Familiarity with GITC is a plus.
  • Set up CI/CD pipelines using Azure DevOps, GitHub Actions, or GitLab CI.
  • Oversee pipeline monitoring with Databricks Observability, Azure Monitor, or Prometheus.
  • Data Governance & Security - Implement RBAC, data lineage, and cataloging through Unity Catalog, Azure Purview, or Collibra.
  • Ensure adherence to GDPR, HIPAA, and internal governance standards.
  • Apply encryption, masking, and audit logging practices.
  • Analytics & Reporting - Create Power BI dashboards and reports that are integrated with Databricks.
  • Facilitate real-time insights and self-service analytics capabilities.
  • Assist with integration into Tableau or Looker as required.
  • Collaboration & Leadership - Collaborate across functions with data scientists, analysts, and business teams.
  • Guide junior engineers and contribute to best practices in Databricks.
  • Communicate effectively in dynamic environments.

Benefits

  • Health, Dental and Vision plans
  • 401(k) Match
  • Volunteer time off
  • Short-term and long-term disability
  • Accident, Life and Travel insurance, as well as flexible spending
  • Tuition Reimbursement Options
  • Employee Assistance Program (EAP)
  • Length of Service Awards
  • Medical/Dental/Vision Insurance
  • 401(k) Retirement Plan - US
  • RRS Plan – CAN
  • Paid Parental Leave
  • Paid Holidays and Paid Time Off
© 2024 Teal Labs, Inc
Privacy PolicyTerms of Service