Senior Data Engineer (Cloud/AWS)
About The Position
What is the opportunity? Supports the development and maintenance of scalable data stores that supply big data in forms needed for business analysis. Applies experience and seasoned knowledge, skills, and practices to perform a variety of assignments. What will you do? Develop and maintain backend systems using Python, PySpark ensuring high performance, scalability, and reliability Participate in the design and implementation of data engineering and ETL pipelines using PySpark Collaborate with cross-functional teams to identify and prioritize project requirements Mentor and guide engineers, providing technical guidance and code reviews Stay up-to-date with the latest technologies and frameworks, and apply this knowledge to improve existing systems and processes Lead the technical direction of the team, including architecture, design, and implementation of software systems Collaborate with product owners to define and prioritize requirements Develop and maintain technical documentation, including architecture diagrams, and technical specifications Participate in code reviews, ensuring high-quality code and adherence to coding standards Collaborate with DevOps and Operations teams to ensure smooth deployment and operation of applications What do you need to succeed?
Requirements
- 5+ years of professional experience with cloud platforms, including Amazon Web Services (AWS), and OCP ( Open Shift )
- 5+ years of experience with PySpark for Data Engineering/ETL pipelines
- Familiarity with containerization using Docker and orchestration using Kubernetes
- Expert in DevOps practices and tools for CI/CD pipelines.
- Willingness to learn new technologies and adapt to new challenges
- Excellent communication and collaboration skills, with the ability to work effectively with cross-functional teams
- Ability to drive technical innovation, including researching new technologies, evaluating technical options, and recommending technical solutions
- Strong problem-solving skills, with the ability to debug complex issues, optimize system performance, and ensure high-quality software deliver
- Experience with Agile development methodologies, including Scrum or Kanban, and the ability to apply these principles to lead the team
Nice To Haves
- Proven experience in leading technical teams, including technical guidance, mentoring, and coaching
- Strong technical leadership skills, including the ability to make technical decisions, prioritize tasks, and manage technical resources
Responsibilities
- Develop and maintain backend systems using Python, PySpark ensuring high performance, scalability, and reliability
- Participate in the design and implementation of data engineering and ETL pipelines using PySpark
- Collaborate with cross-functional teams to identify and prioritize project requirements
- Mentor and guide engineers, providing technical guidance and code reviews
- Stay up-to-date with the latest technologies and frameworks, and apply this knowledge to improve existing systems and processes
- Lead the technical direction of the team, including architecture, design, and implementation of software systems
- Collaborate with product owners to define and prioritize requirements
- Develop and maintain technical documentation, including architecture diagrams, and technical specifications
- Participate in code reviews, ensuring high-quality code and adherence to coding standards
- Collaborate with DevOps and Operations teams to ensure smooth deployment and operation of applications
Benefits
- We thrive on the challenge to be our best, progressive thinking to keep growing, and working together to deliver trusted advice to help our clients thrive and communities prosper.
- We care about each other, reaching our potential, making a difference to our communities, and achieving success that is mutual.
- A comprehensive Total Rewards Program including bonuses and flexible benefits, competitive compensation, commissions, and stock where applicable
- Leaders who support your development through coaching and managing opportunities
- Ability to make a difference and lasting impact
- Work in a dynamic, collaborative, progressive, and high-performing team
- A world-class training program in financial services
- Flexible work/life balance options
- Opportunities to do challenging work
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Mid Level
Education Level
No Education Listed
Number of Employees
5,001-10,000 employees
