Lead Data Engineer
About The Position
The Lead Data Engineer at Northwestern Mutual Life Insurance Company in Milwaukee, Wisconsin will work on development, automation, TOIL elimination, incident response, root cause analysis, and monitoring improvements. Leverage software engineering skillset to improve reliability of infrastructure. Lead efforts to implement site reliability engineering framework for cloud infrastructure. Design and develop integrated monitoring solutions across multiple technical stacks. Build and maintain data storage solutions to aggregate key metrics in an accessible format to display past and current performance. Establish and advocate for CICD best practices for technical stacks in the core platform area. Utilize CICD tooling to create self-healing mechanisms to address system issues and eliminate manual effort. Respond to critical system outages and resolve issues. Use lessons learned from these issues to drive engineering improvement making our systems more resilient. Create, collect, and store data used for system architecture and data flow analysis. Design, build, and maintain robust data pipelines using AWS Glue for Extract, Transform, Load (ETL) operations and AWS Lake Formation for centralized data governance and access control. Develop quality code that leverages existing Glue Terraform Module to automate provisioning of AWS Glue resources, ensuring reusable, version-controlled, and facilitate consistent infrastructure deployment across environments. Continuously improve data processing workflows by identifying bottlenecks and implementing performance. Provide thoughtful system monitoring and resilient architecture. Oversee a team of Data Engineers. Guide and mentor junior team members and direct reports and establish positive relationships with application development customers. Learn new tools and systems as needed. Telecommuting permitted 2 days per week.
Requirements
- Bachelor’s degree in Computer Engineering or a related field
- 5 years of experience as a software engineer or in a related occupation
- 5 years of experience with AWS Glue
- 5 years of experience with AWS Cloud
- 5 years of experience with Terraform
- 5 years of experience with Lake Formation
- 5 years of experience with Databricks
- 5 years of experience with Python programming
- 5 years of experience with Informatica PowerCenter and Power Exchange admin and development
- 5 years of experience with Relational Databases (DB2 LUW, Redshift, MySQL) and shell scripting
- 5 years of experience with DevOps, CICD and Git
- 5 years of experience with Open Table Format (Iceberg & Delta)
- 5 years of experience with Power BI
Responsibilities
- Development, automation, TOIL elimination, incident response, root cause analysis, and monitoring improvements.
- Improve reliability of infrastructure using software engineering skillset.
- Lead efforts to implement site reliability engineering framework for cloud infrastructure.
- Design and develop integrated monitoring solutions across multiple technical stacks.
- Build and maintain data storage solutions to aggregate key metrics in an accessible format to display past and current performance.
- Establish and advocate for CICD best practices for technical stacks in the core platform area.
- Utilize CICD tooling to create self-healing mechanisms to address system issues and eliminate manual effort.
- Respond to critical system outages and resolve issues.
- Use lessons learned from critical system outages to drive engineering improvement making systems more resilient.
- Create, collect, and store data used for system architecture and data flow analysis.
- Design, build, and maintain robust data pipelines using AWS Glue for ETL operations.
- Implement centralized data governance and access control using AWS Lake Formation.
- Develop quality code that leverages existing Glue Terraform Module to automate provisioning of AWS Glue resources.
- Continuously improve data processing workflows by identifying bottlenecks and implementing performance.
- Provide thoughtful system monitoring and resilient architecture.
- Oversee a team of Data Engineers.
- Guide and mentor junior team members and direct reports.
- Establish positive relationships with application development customers.
- Learn new tools and systems as needed.
Benefits
- Salary $156,374- $212,550 per year
- Telecommuting permitted 2 days per week
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
