Lead Data Engineer [Multiple Positions Available]
About The Position
The Lead Data Engineer will be responsible for building and deploying AWS components using industry-leading automation tools. This role involves analyzing, designing, developing, and delivering software solutions in both cloud and on-premises environments. The engineer will solve complex problems across multiple services and cross-functional teams, applying continuous integration and test automation patterns for high-quality production releases. A key aspect of this role is guiding and mentoring junior team members in defining, designing, and developing procedures and solutions that meet business requirements, using industry-leading continuous integration and testing automation practices. The Lead Data Engineer will also collaborate with product partners to drive prioritization decisions and trade-offs, automate and simplify all aspects of software delivery and development, and conduct technical reviews of code to identify anti-patterns and promote continuous refactoring.
Requirements
- Designing scalable serverless cloud-native applications to ingest and transform data using AWS cloud runtime environments including EC2 and Lambda, and big data runtime environments including EMR and Glue.
- Architecting and implementing large scale data migration from big data systems including Hadoop and Hive.
- Using AWS components including S3, Redshift, and Lambda for scalable data storage and processing.
- Leading Agile SDLC projects to design and develop software applications using Python, Java, and Scala, focusing on data processing and analysis.
- Deploying and managing applications on AWS cloud infrastructure.
- Configuring Jenkins pipelines for continuous integration and deployment.
- Implementing comprehensive testing strategies, including automated, functional, performance, and unit testing, while managing code repositories using GIT.
- Working with product partners to drive prioritization decisions and trade-offs.
- Building automation to improve software delivery and development.
- Performing technical review of code across applications and their dependencies to look for antipatterns and promote continuous refactoring.
Responsibilities
- Build and deploy AWS components using industry leading automation tools.
- Analyze, design, develop and deliver software solutions in cloud and on premises.
- Solve complex problems across multiple services and cross-functional teams.
- Apply continuous integration and test automation patterns for delivering high-quality artifacts to production.
- Guide junior team members in defining, designing, and developing procedures and solutions that meet the business requirements.
- Mentor junior team members on development practices and coding guidelines using the industry leading continuous integration and testing automation practices.
- Work with product partners to drive prioritization decisions and trade-offs.
- Automate and simplify all aspects of software delivery and development.
- Undertake a technical review of code across applications and their dependencies to look for anti-patterns and promote continuous refactoring.
Benefits
- comprehensive health care coverage
- on-site health and wellness centers
- a retirement savings plan
- backup childcare
- tuition reimbursement
- mental health support
- financial coaching
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
