Senior Data Engineer

Bank of America•Charlotte, NC

23h

About The Position

At Bank of America, the Senior Data Engineer drives data engineering efforts to deliver enterprise-wide capabilities and complex data solutions. The role directs code design and delivery for integrating, cleaning, transforming, and controlling data in operational and analytical systems, and works with Project Management to define outcomes and work structures. It provides technical thought leadership in implementing complex data solutions spanning multiple systems and domains. The engineer leads requirements analysis, design, development, and implementation of enterprise-scale data platforms and integration solutions. The role calls for seasoned technical leadership, including design reviews, mentoring and coaching development teams, and creating flexible solution designs aligned with the broader portfolio. The engineer develops software that meets functional, non-functional, and compliance requirements, and ensures integration through robust, well-tested interfaces. Multiple projects with competing deadlines may be managed concurrently, with work coordinated across teams and time zones. The role designs and delivers real-time, near-real-time, and batch data integration solutions using technologies such as Scala, Spark, Kafka, Hadoop, Starburst, cloud technologies, object storage, and Informatica. It also coordinates with deployment and support teams to ensure successful testing and production releases. The position is a proficient technical resource working under minimal supervision and typically requires 10 or more years of data delivery or integration experience. The schedule is 1st shift, 40 hours per week.

Requirements

10 or more years of experience in Data Engineering; the description also cites typically 10 or more years of data delivery or integration experience.
Hands-on experience with Scala, Spark, and Kafka Streaming.
Very good understanding of Hadoop components, architecture, and toolsets, including Spark and Scala, Informatica, Starburst, and object storage in enterprise data platforms.
Experience architecting, developing, and implementing solutions using real-time streaming technologies.
Knowledge of performance tuning.
Experience with Exadata and Oracle SQL tuning.
Experience with Starburst and object storage.
Experience developing streaming, near-real-time, and batch data integration solutions.
Hands-on experience with UNIX shell scripting.
Working experience with scheduling tools.
Knowledge of Hadoop Spark/Spark Streaming best practices and standards, as well as standard software development lifecycle experience.
Ability to quickly grasp technology concepts.
Solid quantitative, analytical, process development, facilitation, and organizational skills.
Strong communication, presentation, interpersonal, software development, and work management skills.
Experience developing software that meets functional, non-functional, and compliance requirements and integrates into wider applications or systems through robust, well-tested interfaces.
Ability to lead multiple projects with competing deadlines.
Experience with Agile processes.

Nice To Haves

Experience with large-scale analytical platforms, metadata-driven architectures, and distributed data processing frameworks.
CICD build and deployment tools.
API development experience.
Financial Services or Banking experience.
Experience in distributed computing.
Experience with open table formats such as Apache Iceberg, Delta Lake, or equivalent technologies.
Knowledge of Treasury Payments and Deposits domains.

Responsibilities

Assemble large, complex data sets that meet functional and non-functional requirements, maintaining consistency across systems in design and engineering approaches.
Maintain, improve, clean, and manipulate large data sets for operational and analytical systems; build processes for data transformation, data structures, metadata, data quality controls, dependencies, and workload management.
Communicate information needed for deployment, maintenance, and support of business functionality.
Use multiple architectural components to design and develop client requirements, collaborating with development teams to ensure the data architecture is feasible to implement.
Define and build data pipelines that support data-informed decision-making while adhering to release processes and risk management routines.
Contribute to integration, regression, and performance test suites; analyze test reports, identify issues and errors, and lead triage of underlying causes.
Identify gaps in adherence to data management standards and work with partners to develop plans to close them.
Lead concept testing and research to prototype toolsets and improve existing processes.
Mentor Data Engineers in continuous integration and continuous delivery events, and define key performance indicators and internal controls.
Lead requirements analysis, design, development, and implementation of enterprise-scale data platforms and integration solutions.
Conduct design reviews, mentor and coach development teams, and produce flexible solution designs aligned with the larger portfolio.
Coordinate with deployment and support teams to ensure projects are deployed successfully into testing and production environments.
Lead multiple projects concurrently and coordinate work with teams in different time zones.
Design, develop, and deploy real-time, near-real-time, and batch data integration solutions.