Principal Engineer, Data Warehousing
About The Position
We are seeking a Principal Engineer, Data Warehousing to act as a data engineering expert and partner to other technology teams and data consumers. This role is crucial in controlling the complexity and cost of the data platform while enabling performance, governance, and maintainability. You will understand current and future data consumption patterns, partner with Architects and other technology teams to ensure optimal design of data layers, and collaborate with data governance and information security to evolve data access and permission patterns. The role involves leading research into new technologies like Delta tables and observability solutions, formulating assessment frameworks for new technology, and acting as a best practice expert and blueprint creator for ways of working such as testing, logging, CI/CD, and release lifecycle. You will also manage and provide technical leadership to a small team of data engineers to grow the utilization and impact of the enterprise data platform.
Requirements
- Bachelor’s degree in Computer Science, Computer Engineering or related field of study
- 5 years of experience at a senior level such as Technical Lead or Lead Engineer involving Data Engineering with Design, Architecture, Development and Application Ownership.
- 5 years of experience with writing, optimizing and analyzing SQLs
- 5 years of experience with interpreting complex data requirements and architect solutions
- 5 years of experience with programming in Shell, Perl or Python
- 5 years of experience with Spark, MPP databases, ETL tools and data warehouse
- 5 years of experience with CI/CD with bitbucket or GitHub
- 5 years of experience working closely with a business team
- 5 years of experience with Software Development Life Cycle (SDLC) processes and workflow covering both Agile and Waterfall methodologies
- 5 years of experience with Data Engineering on large scale platforms
- 5 years of experience leading multiple projects, colleagues and execution partners from a technology perspective
- 5 years of experience with Application Management
- 5 years of experience leading Data Migration projects
Responsibilities
- Act as a data engineering expert and partner to other technology teams and data consumers in controlling complexity and cost of the data platform, whilst enabling performance, governance, and maintainability of the estate.
- Understand current and future data consumption patterns, partner with Architects and other technology teams to ensure optimal design of data layers.
- Collaborate with data governance and information security in evolving data access and permission patterns to enable optimal consumer experience that is secure and compliant with the enterprise standards and audit requirements.
- Lead and hands-on execution of research into new technologies such as Delta tables, observability solutions.
- Formulate frameworks for assessment of new technology against business benefit, implications for data consumers.
- Act as a best practice expert, blueprint creator of ways of working such as testing, logging, CI/CD, release lifecycle, enabling rapid growth in data inventory and utilization of Data Science Platform.
- Manage and provide technical leadership to a small team (1-3 initially) of data engineers in growing utilization and impact of the enterprise data platform offering.
Benefits
- Internal Referrals for this position are eligible for the Employee Referral Program.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
