Principal Data Engineer

Fidelity Investments•Durham, NC

2d•Hybrid

About The Position

Position Description: Develops enterprise data warehouse and reporting platforms using Oracle SQL and Amazon Web Services (AWS) -- S3, Aurora PostgresSQL, and Lambda. Migrates legacy on-premise databases to AWS Aurora PostgresSQL and Snowflake databases. Develops orchestration processes using Control-M or Autosys platforms. Builds and deploys applications using Continuous Integration (CI) pipelines – GitHub, Liquibase, Alembic, and AWS CodeCommit. Performs unit and integration testing using SQL queries, Python, Spark, and PySpark. Provides Extract Transform and Load (ETL) solutions by developing complex or multiple software applications. Develops reporting platform using Business Objects, Tableau, or PowerBI.

Requirements

Bachelor’s degree in Computer Science, Engineering, Information Technology, Information Systems, or a closely related field (or foreign education equivalent) and five (5) years of experience as a Principal Data Engineer (or closely related occupation) performing data analysis, data modeling, database tuning, Change Data Capture (CDC), and implementing data solutions in AWS and on prem data centers, using ETL tools.
Or, alternatively, Master’s degree in Computer Science, Engineering, Information Technology, Information Systems, or a closely related field (or foreign education equivalent) and three (3) years of experience as a Principal Data Engineer (or closely related occupation) performing data analysis, data modeling, database tuning, Change Data Capture (CDC), and implementing data solutions in AWS and on prem data centers, using ETL tools.
Demonstrated Expertise (“DE”) performing data modeling, design, and development of Online Transaction Processing (OLTP)/Online Analytical Processing (OLAP) databases -- Oracle, Sybase, AWS RDS Postgres, MYSQL, and Databricks -- for enterprise or financial services, using SQL, PL/SQL, and Python.
DE building big data processing architecture using AWS Cloud Services -- S3, Lambda, EC2, EMR, IAM, KMS, Step Function, MWAA, Python CDK, and DMS; implementing ETL frameworks (Databricks Auto Loader) and Delta Live Tables (DLT) using orchestrator tools -- Autosys, Airflow, and Control-M; and performing monitoring of all data infrastructure using Grafana.
DE designing, developing, and supporting reporting and analytical dashboards to meet the needs of client service users and data scientists, using Tableau.
DE automating the deployment of database and applications through established Continuous Integration/Continuous Deployment (CI/CD) pipelines, using GitHub, Bitbucket, AWS Code Commit, Alembic, and Jenkins.

Responsibilities

Designs, implements, and maintains data structures, batch jobs, and interfaces to external systems.
Develops original and creative technical solutions to on-going development efforts.
Develops applications for multiple projects supporting several divisional initiatives.
Supports and performs all phases of testing leading to implementation.
Assists in the planning and conducting of user acceptance testing.
Develops comprehensive documentation for multiple applications supporting several corporate initiatives.
Responsible for post-installation validation and triaging of any issues.
Establishes project plans for projects of moderate scope.
Performs independent and complex technical and functional analysis for multiple projects supporting several initiatives.
Manages data services hosted on the operational data stores and file-based interfaces.
Confers with systems analysts and other software engineers/developers to design systems.
Gathers information on project limitations and capabilities, performance requirements, and interfaces.
Develops and oversees software system testing and validation procedures, programming, and documentation.