Staff Engineer (e-ST)

Thrivent•Minneapolis, MN

2d•$173,400•Hybrid

About The Position

Design, develop, and maintain data pipelines to handle large and complex data sets using Big Data technologies such as Hadoop, Spark, and Kafka. Ensure that data pipelines are scalable, reliable, and performant, and that they adhere to data governance, data quality best practices. Help develop sustainable data solutions with current and leading next generation data technologies to transform business capabilities. Collaborate with end users to ensure data availability and accessibility. Work with cross-functional teams to identify and implement process improvements for data pipelines and data workflows. Monitor and troubleshoot data pipeline issues to ensure high availability and reliability. Participate in code reviews and provide feedback to other team members to ensure that code is high quality and adheres to best practices. Remain current with industry trends and emerging technologies in Big Data and data engineering and evaluate their potential impact on our data infrastructure and systems. Lead, mentor, and develop up and coming engineers. Engage in technology strategy and data architecture across enterprise.

Requirements

Minimum of a Bachelor’s or foreign equivalent degree in Management Information Systems, Computer Science, or related field plus 6 years of progressively responsible post baccalaureate experience as a data engineer.
At least 6 years of Data Engineering experience with Big Data Technologies: Apache Spark and Hadoop.
At least 5 years of experience with Programming with Python, Java or Scala.
At least 5 years of experience Designing and architecting big data solutions using Spark and Hadoop.
At least 4 years of experience Building data pipelines, CICD pipelines, and fit for purpose data stores.
At least 4 years of experience with BI development and reports.
At least 4 years of experience with Data Warehouse modelling with star schema.
At least 4 years of experience modelling data in big data eco system.
At least 4 years of experience Developing and executing test cases.
At least 3 years of experience building data pipelines using Snowflake, Databricks and Kafka.
At least 2 years of experience with Streamsets and Bamboo.
At least 2 years of experience Working with Informatica, IICS and DB2.
At least 2 years of experience with Cloud technologies: AWS, Azure, Google Cloud, OpenStack, Docker, Ansible, Chef, Lambdas, Microservices and Terraform.

Responsibilities

Design, develop, and maintain data pipelines to handle large and complex data sets using Big Data technologies such as Hadoop, Spark, and Kafka.
Ensure that data pipelines are scalable, reliable, and performant, and that they adhere to data governance, data security, and data quality best practices.
Help develop sustainable data solutions with current and leading next generation data technologies to transform business capabilities.
Collaborate with end users to ensure data availability and accessibility.
Work with cross-functional teams to identify and implement process improvements for data pipelines and data workflows.
Monitor and troubleshoot data pipeline issues to ensure high availability and reliability.
Participate in code reviews and provide feedback to other team members to ensure that code is high quality and adheres to best practices.
Remain current with industry trends and emerging technologies in Big Data and data engineering and evaluate their potential impact on our data infrastructure and systems.
Lead, mentor, and develop up and coming engineers.
Engage in technology strategy and data architecture across enterprise.