Big Data Engineer

Citi • Irving, TX

1d

About The Position

Design and implement scalable and efficient Hadoop architecture solutions. Collaborate with data engineers and scientists to understand data requirements and translate them into robust data solutions. Optimize Hadoop clusters for performance and resource utilization, ensuring efficiency and cost-effectiveness. Maintain and monitor Hadoop infrastructure, ensuring high availability, reliability, and disaster recovery capabilities. Implement data security and governance policies to protect sensitive information and ensure compliance with regulatory standards. Stay updated with the latest advancements in Hadoop and big data technologies, evaluating and recommending new tools and practices. Troubleshoot and resolve complex issues within the Hadoop ecosystem, minimizing downtime and ensuring continuous operations. Responsible for developing Spark-based solutions to support near real-time data ingestion, analytics, and reporting needs. 3-8 years of relevant experience in the Financial Service industry. Consistently demonstrates clear and concise written and verbal communication. Demonstrated problem-solving and decision-making skills. Ability to work under pressure and manage deadlines or unexpected changes in expectations or requirements. Big Data Technologies: Hadoop HDFS (Hadoop Distributed File System) YARN (Yet Another Resource Negotiator) MapReduce Hive Spark Proven experience in designing and managing Hadoop-based architectures. Strong understanding of Hadoop ecosystem components. Strong hands-on and architectural knowledge of Python, PySpark, and Unix. Exposure to AI/ML lifecycle management, MLOps, and GenAI solutions. Bachelor's degree/University degree or equivalent experience

Requirements

• 3-8 years of relevant experience in the Financial Service industry.
• Consistently demonstrates clear and concise written and verbal communication.
• Demonstrated problem-solving and decision-making skills.
• Ability to work under pressure and manage deadlines or unexpected changes in expectations or requirements.
• Big Data Technologies: Hadoop HDFS (Hadoop Distributed File System) YARN (Yet Another Resource Negotiator) MapReduce Hive Spark
• Proven experience in designing and managing Hadoop-based architectures.
• Strong understanding of Hadoop ecosystem components.
• Strong hands-on and architectural knowledge of Python, PySpark, and Unix.
• Bachelor's degree/University degree or equivalent experience

Nice To Haves

• Exposure to AI/ML lifecycle management, MLOps, and GenAI solutions.

Responsibilities

• Design and implement scalable and efficient Hadoop architecture solutions.
• Collaborate with data engineers and scientists to understand data requirements and translate them into robust data solutions.
• Optimize Hadoop clusters for performance and resource utilization, ensuring efficiency and cost-effectiveness.
• Maintain and monitor Hadoop infrastructure, ensuring high availability, reliability, and disaster recovery capabilities.
• Implement data security and governance policies to protect sensitive information and ensure compliance with regulatory standards.
• Stay updated with the latest advancements in Hadoop and big data technologies, evaluating and recommending new tools and practices.
• Troubleshoot and resolve complex issues within the Hadoop ecosystem, minimizing downtime and ensuring continuous operations.
• Responsible for developing Spark-based solutions to support near real-time data ingestion, analytics, and reporting needs.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Number of Employees: 5,001-10,000 employees