Senior Data Engineer

EMC Insurance Companies•Des Moines, IA

11d•$103,954 - $164,309•Hybrid

About The Position

At EMC, we’re all about working together to make an impact. As part of our team, you’ll have the opportunity to grow, contribute, and gain experience that matters. We strive to be caring leaders, close partners, and responsive experts—always supporting each other to do our best work. Join us, and let’s improve lives together. This position is eligible to work a hybrid schedule within the Des Moines, Iowa Metro Area E ssential Functions: Develops, test, and maintains prioritized engineering (design/build/test) work as a member of a data analytics product pipeline architecture and infrastructure team: Ensures delivery of solutions for organization of data, such as transformation, conformation, integration, and aggregation of data Partners with the Data Platform Operations (DPOs), Information Security, Network, and Operations teams to design and implement new data pipelines for new business initiatives as related to data and analytics Serves as a technical expert for the Data Engineering team Ensures best practices and internal processes are followed when solutions are designed and implemented Ensures the delivery of solutions for acquisition of data, such as landing, staging, rapid ingestion, and historical data extraction Creates processes framework, design and implementation of large-scale data conversion and data ingestion on various cloud and on-premise platforms using most optimal processing and programming techniques Investigates and analyzes feasibility of data integration prototypes and program requirements Develops specifications and plans the full range of programming actions to produce data integration components Develops and maintains highly complex automated ETL/ELT pipeline architecture that cleans, transforms, and aggregates data from disparate sources using architecturally approved tools, programming languages, and services to deliver a data mesh approach Develops and maintains highly complex automated ETL/ELT monitoring and alarming solutions using architecturally approved languages and services Monitors, troubleshoots, and optimizes data pipelines for quality, performance, and reliability creating a single source of truth providing business customers with highly reliable and quality data Builds highly complex unit tests to enable test driven development and automation to drive code reliability Develops, maintains, tests, and deploys code with architecturally approved toolsets using Dev Ops practices, ETL/ELT pipelines, data catalogs and data queries Serves as a technical resource and supports team members on EMC’s data integration projects and helps them learn new technologies Leads evaluation of new technologies including performing POCs and presenting results to others, with a goal of providing technical recommendations Balances and designs data analytics product solutions based on cost and performance to meet business requirements Guides business customers to develop, troubleshoot, and optimize complex SQL and ETL/ELT solutions to solve reporting, metrics, and analytics problems. Data analysis: Collaborates with cross-functional teams to understand data requirements for data analytics products and translate them into scalable and efficient data solutions: Leads complex projects and provides subject matter expertise, support and advice to other projects being handled by the team Partners with users and stakeholders to understand data needs and business requirements Leads the team on conceptualizing and developing new data solutions to meet the business data needs and requirements Researches, performs analysis, and proposes effective solutions related to system developments and enhancements through collaboration with Data Architecture Analyzes and reviews potential adjustments or modifications for impacts on other programs Drives best practices in data governance, data security, and privacy principles to support data lineage and reference data needs. Collaborates with cross-functional teams to understand business and downstream application (BI reporting, Data Science, Data Governance, other System integrations) and consumer requirements to deliver data-driven solutions that implement best practices for data management, security, and governance Data modeling: Develops highly complex data objects for business analytics using data modeling techniques Models front-end and back-end data sources to help draw a more comprehensive picture of user flows throughout the system and to enable powerful data analysis Works with stakeholders to understand highly complex business needs and translates them into a conceptual data model, defining entities, attributes, and relationships Builds highly complex detailed logical models based on the conceptual model, specifying data types, constraints, and keys to ensure data integrity Implements highly complex logical models into appropriate database management system, considering database features, storage optimization, and performance considerations Designs highly complex data pipelines to extract data from source systems, transform it as needed based on the data model, and load it into the target database Documentation: Participates in creating and maintaining documentation related to the design of applications and interfaces to enterprise and third-party systems in accordance with departmental and team standards Collaborates with data governance on ensuring what is being produced in data analytics products enables data governance to publish a data asset Collaborates with data governance to develop and maintain data analytic products, artifacts and user documentation for business customers Works with business customers and software development teams to gather and document requirements for data publishing and data consumption via data mesh architecture. Maintains systems documentation

Requirements

Bachelor’s degree, preferably in a computer related field or equivalent relevant experience
Eight years of data integration development experience, or related experience
Excellent SQL Knowledge and experience working with relational databases
Excellent analytics skills to work with unstructured datasets preferred
Excellent knowledge of applicable programming languages such as Python
Strong knowledge of cloud services such as RDS, S3, SNS, CloudWatch and Lambda
Advanced knowledge of DBT/Informatica or other ETL/ELT tools preferred
Excellent knowledge of Snowflake
Excellent knowledge of data modeling
Excellent ability to build processes supporting data transformation, data structures, metadata, dependency and workload management
Excellent knowledge of orchestration or scheduling tools
Excellent ability to manipulate, process and extract value from large, disconnected datasets
Advanced knowledge of DevOps and/or DataOps practices, CI/CD pipelines, and containerization technologies preferred
Excellent verbal and written communication skills
Strong attention to detail, organizational and multi-tasking skills required with the ability to adapt to changing priorities
Ability to maintain confidentiality

Responsibilities

Develops, test, and maintains prioritized engineering (design/build/test) work as a member of a data analytics product pipeline architecture and infrastructure team
Ensures delivery of solutions for organization of data, such as transformation, conformation, integration, and aggregation of data
Partners with the Data Platform Operations (DPOs), Information Security, Network, and Operations teams to design and implement new data pipelines for new business initiatives as related to data and analytics
Serves as a technical expert for the Data Engineering team
Ensures best practices and internal processes are followed when solutions are designed and implemented
Ensures the delivery of solutions for acquisition of data, such as landing, staging, rapid ingestion, and historical data extraction
Creates processes framework, design and implementation of large-scale data conversion and data ingestion on various cloud and on-premise platforms using most optimal processing and programming techniques
Investigates and analyzes feasibility of data integration prototypes and program requirements
Develops specifications and plans the full range of programming actions to produce data integration components
Develops and maintains highly complex automated ETL/ELT pipeline architecture that cleans, transforms, and aggregates data from disparate sources using architecturally approved tools, programming languages, and services to deliver a data mesh approach
Develops and maintains highly complex automated ETL/ELT monitoring and alarming solutions using architecturally approved languages and services
Monitors, troubleshoots, and optimizes data pipelines for quality, performance, and reliability creating a single source of truth providing business customers with highly reliable and quality data
Builds highly complex unit tests to enable test driven development and automation to drive code reliability
Develops, maintains, tests, and deploys code with architecturally approved toolsets using Dev Ops practices, ETL/ELT pipelines, data catalogs and data queries
Serves as a technical resource and supports team members on EMC’s data integration projects and helps them learn new technologies
Leads evaluation of new technologies including performing POCs and presenting results to others, with a goal of providing technical recommendations
Balances and designs data analytics product solutions based on cost and performance to meet business requirements
Guides business customers to develop, troubleshoot, and optimize complex SQL and ETL/ELT solutions to solve reporting, metrics, and analytics problems.
Collaborates with cross-functional teams to understand data requirements for data analytics products and translate them into scalable and efficient data solutions
Leads complex projects and provides subject matter expertise, support and advice to other projects being handled by the team
Partners with users and stakeholders to understand data needs and business requirements
Leads the team on conceptualizing and developing new data solutions to meet the business data needs and requirements
Researches, performs analysis, and proposes effective solutions related to system developments and enhancements through collaboration with Data Architecture
Analyzes and reviews potential adjustments or modifications for impacts on other programs
Drives best practices in data governance, data security, and privacy principles to support data lineage and reference data needs.
Collaborates with cross-functional teams to understand business and downstream application (BI reporting, Data Science, Data Governance, other System integrations) and consumer requirements to deliver data-driven solutions that implement best practices for data management, security, and governance
Develops highly complex data objects for business analytics using data modeling techniques
Models front-end and back-end data sources to help draw a more comprehensive picture of user flows throughout the system and to enable powerful data analysis
Works with stakeholders to understand highly complex business needs and translates them into a conceptual data model, defining entities, attributes, and relationships
Builds highly complex detailed logical models based on the conceptual model, specifying data types, constraints, and keys to ensure data integrity
Implements highly complex logical models into appropriate database management system, considering database features, storage optimization, and performance considerations
Designs highly complex data pipelines to extract data from source systems, transform it as needed based on the data model, and load it into the target database
Participates in creating and maintaining documentation related to the design of applications and interfaces to enterprise and third-party systems in accordance with departmental and team standards
Collaborates with data governance on ensuring what is being produced in data analytics products enables data governance to publish a data asset
Collaborates with data governance to develop and maintain data analytic products, artifacts and user documentation for business customers
Works with business customers and software development teams to gather and document requirements for data publishing and data consumption via data mesh architecture.
Maintains systems documentation