Data Engineer

Delta Dental • Alpharetta, GA

3d • Hybrid

About The Position

Delta Dental Insurance Company is seeking a Data Engineer to design, build, and optimize data pipelines and workflows. This role involves integrating data from various sources into the Enterprise Data Platform (EDP), working with Snowflake, Matillion, Azure Data Factory, and BI tools. The Data Engineer will also be involved in master data management using EBX, implementing data governance, optimizing database systems, and collaborating with cross-functional teams. The position requires mentoring associate data engineers, supporting data scientists and analysts, and developing data models and documentation. Collaboration with infrastructure and DevOps teams for system security and reliability is key. The role also includes data profiling, analysis, staying updated with new technologies, troubleshooting data issues, and ensuring compliance with data privacy regulations. Performance monitoring, capacity planning, code reviews, and participation in data architecture discussions are also part of the responsibilities. The work is performed in an Agile development environment with CI/CD best practices, focusing on continuous monitoring and optimization of data engineering processes for efficiency, reliability, and scalability.

Requirements

• Bachelor’s degree or foreign degree equivalent in Computer Science or related field and five (5) years of progressive, post-baccalaureate experience in Data Integration and ETL development, Cloud Data Platforms, Data Engineering, Data Analytical Models, Data Governance, Power BI reporting and EBX for enterprise data management or related roles in the job offered.
• Design and build scalable data pipelines using Matillion for Enterprise Data Platforms.
• Design and implement data models and curated datasets on Snowflake to enable analytics, reporting, and business insights.
• Configure and maintain EBX data models to create and manage business workflows and implement data governance rules to support enterprise-wide master data management.
• Collibra to manage data definitions, lineage, ownership, and data quality rules and ensure data assets are properly documented, governed, and compliant with enterprise standards.
• Power BI to build dashboards, scorecards, and analytical insights that support business decision-making and enable enterprise reporting and self-service analytics.
• Experience with data science to apply scientific methods, statistical techniques, and algorithms to extract insights from structured and unstructured data.
• Python to develop data engineering solutions, automation frameworks, and advanced data processing workflows ensuring enhanced scalability, reliability, and efficiency across Enterprise Data Platforms.
• Must live within reasonable commuting distance from HQ and able to appear in office as required.

Nice To Haves

• Hands on experience with Snowflake including data modelling, data lake, Lakehouse concepts, warehouse management, and performance tuning.
• Experience with Matillion and ADF (Azure Data Factory).
• Experience with BI and reporting tools (e.g., Tableau, Power BI).
• Experience with version control (Git), DevOps, and CI/CD processes.
• Work with EBX for master data management (MDM), ensuring consistent and governed data across systems.
• Implement data governance and quality control measures to ensure data accuracy, consistency, and integrity throughout the data lifecycle.
• Optimize and tune database systems and data processing frameworks for performance and scalability.
• Develop and maintain data ingestion frameworks for both batch and near real-time data processing.
• Mentor and provide guidance to associate data engineers, assisting in their professional development.
• Collaborate with data scientists and analysts to support their data needs, providing efficient access to relevant datasets.
• Develop and maintain data models, schemas, and documentation to facilitate data understanding and accessibility.
• Work closely with infrastructure and DevOps teams to ensure data systems are secure, reliable, and properly integrated with existing infrastructure.
• Perform data profiling, analysis, and data mining tasks to identify patterns, trends, and insights in large datasets.
• Stay up-to-date with emerging data engineering technologies and best practices and proactively recommend improvements to existing systems.
• Troubleshoot and resolve data-related issues, ensuring minimal disruption to data pipelines and systems.
• Collaborate with data privacy and compliance teams to ensure adherence to data privacy regulations and industry standards.
• Conduct performance monitoring and capacity planning to anticipate and address future data engineering needs.
• Participate in code reviews, performance tuning, and data pipeline optimization for efficiency and scalability.
• Participate in data architecture discussions and provide input on data storage, data flow, and data integration strategies.
• Work in an Agile development environment with continuous integration and deployment (CI/CD) best practices.
• Continuously monitor and optimize data engineering processes, striving for efficiency, reliability, and scalability.

Responsibilities

• Design, build, and optimize data pipelines and workflows to integrate data from multiple on-prem and cloud-based sources into the Enterprise Data Platform (EDP).
• Hands on experience with Snowflake including data modelling, data lake, Lakehouse concepts, warehouse management, and performance tuning.
• Experience with Matillion and ADF (Azure Data Factory).
• Experience with BI and reporting tools (e.g., Tableau, Power BI).
• Experience with version control (Git), DevOps, and CI/CD processes.
• Work with EBX for master data management (MDM), ensuring consistent and governed data across systems.
• Collaborate with cross-functional teams to understand data requirements, identify opportunities for data integration, and propose efficient data engineering solutions.
• Implement data governance and quality control measures to ensure data accuracy, consistency, and integrity throughout the data lifecycle.
• Optimize and tune database systems and data processing frameworks for performance and scalability.
• Develop and maintain data ingestion frameworks for both batch and near real-time data processing.
• Mentor and provide guidance to associate data engineers, assisting in their professional development.
• Collaborate with data scientists and analysts to support their data needs, providing efficient access to relevant datasets.
• Develop and maintain data models, schemas, and documentation to facilitate data understanding and accessibility.
• Work closely with infrastructure and DevOps teams to ensure data systems are secure, reliable, and properly integrated with existing infrastructure.
• Perform data profiling, analysis, and data mining tasks to identify patterns, trends, and insights in large datasets.
• Stay up-to-date with emerging data engineering technologies and best practices and proactively recommend improvements to existing systems.
• Troubleshoot and resolve data-related issues, ensuring minimal disruption to data pipelines and systems.
• Collaborate with data privacy and compliance teams to ensure adherence to data privacy regulations and industry standards.
• Conduct performance monitoring and capacity planning to anticipate and address future data engineering needs.
• Participate in code reviews, performance tuning, and data pipeline optimization for efficiency and scalability.
• Participate in data architecture discussions and provide input on data storage, data flow, and data integration strategies.
• Work in an Agile development environment with continuous integration and deployment (CI/CD) best practices.
• Continuously monitor and optimize data engineering processes, striving for efficiency, reliability, and scalability.

Benefits

• Base Pay Information The national base pay range at the end is a good-faith estimate of what Delta Dental may pay for new hires. Actual pay may vary based on Delta Dental’s assessment of the candidate’s knowledge, skills, abilities (KSAs), related experience, education, certifications and ability to meet required minimum job qualifications. Other factors impacting pay include prevailing wages in the work location and internal equity. Pay Grade 21.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level