Data Engineer

U.S. Bank • Irving, TX

1d • Onsite

About The Position

At U.S. Bank, we’re on a journey to do our best. Helping the customers and businesses we serve to make better and smarter financial decisions and enabling the communities we support to grow and succeed. We believe it takes all of us to bring our shared ambition to life, and each person is unique in their potential. A career with U.S. Bank gives you a wide, ever-growing range of opportunities to discover what makes you thrive at every stage of your career. Try new things, learn new skills and discover what you excel at—all from Day One. Job Description Design and implement scalable data lake solutions using Snowflake and Databricks, Develop and optimize data pipelines for ingestion, transformation, and storage. Manage data governance, quality, and security across cloud environments and implement performance tuning, automation, and CI/CD for data workflows. Collaborate with cross-functional teams to support cloud migration activities Performance Optimization Tune Hadoop, Hive, and Spark jobs and configurations for optimal performance, efficiency, and resource utilization. This includes optimizing queries, managing partitions, and leveraging in-memory capabilities. Troubleshooting and Support Diagnose and resolve issues related to Linux servers, networks, cluster health, job failures, and performance bottlenecks. Provide on-call support and collaborate with other teams to ensure smooth operations. Security, Governance, and Secrets Management Implement and manage security measures within the Cloudera environment, including Kerberos, Apache Ranger, and Atlas, to ensure data governance and compliance. Setup and manage HashiCorp Vault for secure keys and secrets management. Utilize CyberArk for privileged access management and secure administrative tasks on the cluster. Data and Application Migration Migrate Datastage ETL jobs to Azure cloud services such as Azure Synapse Analytics, Azure Databricks, or Snowflake. Ensure data integrity, performance tuning, and validation. Automation and Scripting: Develop scripts (e.g., shell, Ansible, Python) for automating administrative tasks, deployments, and monitoring. Work with users to develop, debug, optimize Hive/Spark/Python programs that connect to the Cloudera environment. Documentation Create and maintain documentation for system configurations, operational procedures, and troubleshooting knowledge bases. Vendor Collaboration Work closely with the vendor to stay current with the latest releases, perform upgrades, and address vulnerabilities. Basic Qualifications Bachelor’s degree, or equivalent work experience Three to five years of relevant experience Experience Should Include Deep expertise in Data Engineering and Management technologies, synthetic data, automation, advanced analytics. Ability to do on-call rotation once a month Very strong customer-centric focus 6 - 8 years of hands-on experience in Data engineering, Cloud platform management, and performance optimization. Very strong Azure Data Factory tools experience Excellent SQL Experience, including performance tuning and optimization. Hands-on experience with Hadoop, Hive, Spark, and migration of Big Data into Azure cloud services. DataStage experience for conversion of ETL jobs to Pyspark ETL pipelines Working with offshore teams Working knowledge and hands-on experience in Data Integration and Data Lake Architectures with Databricks and Snowflake platforms. Working knowledge of Microsoft Azure cloud and big data migration to cloud platforms. Proficiency in Linux, clustering, and distributed systems. Expertise in Hive and Spark for data processing and analytics. Expertise in Hadoop ecosystem components such as HDFS, YARN, Hive, Spark, and Sqoop. Proficiency in languages such as shell, Python, Pyspark for automating workflows, deployments, and monitoring. Expertise in Linux, Network, Python scripting, DNS, Kerberos, LDAP/AD, JupyterHub. Experience in creating and maintaining documentation for system configurations, operational procedures, and troubleshooting knowledge bases. Strong problem-solving skills and the ability to diagnose and resolve system failures and performance bottlenecks. Excellent communication and collaboration skills to work effectively with cross-functional teams Preferred Skills and Experience Strong problem-solving, analytical skills. Deep understanding of banking, operations, risk, and compliance—pairing technical fluency with business and regulatory acumen. Experience on change management process and addressing PCI and PII data requirements. Experience in implementing secure systems with secure API design, IAM integration. This role requires working from a U.S. Bank location three (3) or more days per week. If there’s anything we can do to accommodate a disability during any portion of the application or hiring process, please refer to our disability accommodations for applicants. Benefits: Our approach to benefits and total rewards considers our team members’ whole selves and what may be needed to thrive in and outside work. That's why our benefits are designed to help you and your family boost your health, protect your financial security and give you peace of mind. Our benefits include the following: Healthcare (medical, dental, vision) Basic term and optional term life insurance Short-term and long-term disability Pregnancy disability and parental leave 401(k) and employer-funded retirement plan Paid vacation (from two to five weeks depending on salary grade and tenure) Up to 11 paid holiday opportunities Adoption assistance Sick and Safe Leave accruals of one hour for every 30 worked, up to 80 hours per calendar year unless otherwise provided by law Review our full benefits available by employment status here. U.S. Bank is an equal opportunity employer. We consider all qualified applicants without regard to race, religion, color, sex, national origin, age, sexual orientation, gender identity, disability or veteran status, and other factors protected under applicable law. E-Verify U.S. Bank participates in the U.S. Department of Homeland Security E-Verify program in all facilities located in the United States and certain U.S. territories. The E-Verify program is an Internet-based employment eligibility verification system operated by the U.S. Citizenship and Immigration Services. Learn more about the E-Verify program. The salary range reflects figures based on the primary location, which is listed first. The actual range for the role may differ based on the location of the role. In addition to salary, U.S. Bank offers a comprehensive benefits package, including incentive and recognition programs, equity stock purchase 401(k) contribution and pension (all benefits are subject to eligibility requirements). Pay Range: $98,175.00 - $115,500.00 U.S. Bank will consider qualified applicants with arrest or conviction records for employment. U.S. Bank conducts background checks consistent with applicable local laws, including the Los Angeles County Fair Chance Ordinance and the California Fair Chance Act as well as the San Francisco Fair Chance Ordinance. U.S. Bank is subject to, and conducts background checks consistent with the requirements of Section 19 of the Federal Deposit Insurance Act (FDIA). In addition, certain positions may also be subject to the requirements of FINRA, NMLS registration, Reg Z, Reg G, OFAC, the NFA, the FCPA, the Bank Secrecy Act, the SAFE Act, and/or federal guidelines applicable to an agreement, such as those related to ethics, safety, or operational procedures. Applicants must be able to comply with U.S. Bank policies and procedures including the Code of Ethics and Business Conduct and related workplace conduct and safety policies. Posting may be closed earlier due to high volume of applicants. At U.S. Bank, we’re on a journey to do our best. Helping the customers and businesses we serve to make better and smarter financial decisions and enabling the communities we support to grow and succeed. We believe it takes all of us to bring our shared ambition to life, and each person is unique in their potential. A career with U.S. Bank gives you a wide, ever-growing range of opportunities to discover what makes you thrive at every stage of your career. Try new things, learn new skills and discover what you excel at—all from Day One.

Requirements

• Bachelor’s degree, or equivalent work experience
• Three to five years of relevant experience
• Experience Should Include Deep expertise in Data Engineering and Management technologies, synthetic data, automation, advanced analytics.
• Ability to do on-call rotation once a month
• Very strong customer-centric focus
• 6 - 8 years of hands-on experience in Data engineering, Cloud platform management, and performance optimization.
• Very strong Azure Data Factory tools experience
• Excellent SQL Experience, including performance tuning and optimization.
• Hands-on experience with Hadoop, Hive, Spark, and migration of Big Data into Azure cloud services.
• DataStage experience for conversion of ETL jobs to Pyspark ETL pipelines
• Working with offshore teams
• Working knowledge and hands-on experience in Data Integration and Data Lake Architectures with Databricks and Snowflake platforms.
• Working knowledge of Microsoft Azure cloud and big data migration to cloud platforms.
• Proficiency in Linux, clustering, and distributed systems.
• Expertise in Hive and Spark for data processing and analytics.
• Expertise in Hadoop ecosystem components such as HDFS, YARN, Hive, Spark, and Sqoop.
• Proficiency in languages such as shell, Python, Pyspark for automating workflows, deployments, and monitoring.
• Expertise in Linux, Network, Python scripting, DNS, Kerberos, LDAP/AD, JupyterHub.
• Experience in creating and maintaining documentation for system configurations, operational procedures, and troubleshooting knowledge bases.
• Strong problem-solving skills and the ability to diagnose and resolve system failures and performance bottlenecks.
• Excellent communication and collaboration skills to work effectively with cross-functional teams

Nice To Haves

• Strong problem-solving, analytical skills.
• Deep understanding of banking, operations, risk, and compliance—pairing technical fluency with business and regulatory acumen.
• Experience on change management process and addressing PCI and PII data requirements.
• Experience in implementing secure systems with secure API design, IAM integration.

Responsibilities

• Design and implement scalable data lake solutions using Snowflake and Databricks
• Develop and optimize data pipelines for ingestion, transformation, and storage.
• Manage data governance, quality, and security across cloud environments and implement performance tuning, automation, and CI/CD for data workflows.
• Collaborate with cross-functional teams to support cloud migration activities
• Tune Hadoop, Hive, and Spark jobs and configurations for optimal performance, efficiency, and resource utilization. This includes optimizing queries, managing partitions, and leveraging in-memory capabilities.
• Diagnose and resolve issues related to Linux servers, networks, cluster health, job failures, and performance bottlenecks. Provide on-call support and collaborate with other teams to ensure smooth operations.
• Implement and manage security measures within the Cloudera environment, including Kerberos, Apache Ranger, and Atlas, to ensure data governance and compliance. Setup and manage HashiCorp Vault for secure keys and secrets management. Utilize CyberArk for privileged access management and secure administrative tasks on the cluster.
• Migrate Datastage ETL jobs to Azure cloud services such as Azure Synapse Analytics, Azure Databricks, or Snowflake. Ensure data integrity, performance tuning, and validation.
• Develop scripts (e.g., shell, Ansible, Python) for automating administrative tasks, deployments, and monitoring. Work with users to develop, debug, optimize Hive/Spark/Python programs that connect to the Cloudera environment.
• Create and maintain documentation for system configurations, operational procedures, and troubleshooting knowledge bases.
• Work closely with the vendor to stay current with the latest releases, perform upgrades, and address vulnerabilities.

Benefits

• Healthcare (medical, dental, vision)
• Basic term and optional term life insurance
• Short-term and long-term disability
• Pregnancy disability and parental leave
• 401(k) and employer-funded retirement plan
• Paid vacation (from two to five weeks depending on salary grade and tenure)
• Up to 11 paid holiday opportunities
• Adoption assistance
• Sick and Safe Leave accruals of one hour for every 30 worked, up to 80 hours per calendar year unless otherwise provided by law

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level