Big Data DevOps Engineer

VIRTUSA • Pittsburgh, PA

69d

About The Position

We are seeking an experienced Big Data DevOps Engineer with 8Plus years of expertise in designing, building, and supporting large-scale data processing systems. The ideal candidate will have strong hands-on experience with HDFS, Hive, Spark, and Linux environments, and will be responsible for developing robust ETL pipelines and providing end-to-end production support.

Requirements

• Strong hands-on experience in Big Data tools (HDFS, Hive, Spark), ETL pipeline development, and production support in a Linux-based DevOps environment.

Nice To Haves

• Excellent verbal and written communication skills to effectively liaise between business stakeholders and IT teams.
• Strong problem-solving abilities to identify and implement system improvements.
• Attention to detail with the ability to analyze complex business processes and technical requirements.
• Ability to work independently and manage multiple priorities efficiently.
• Strong interpersonal skills for collaborating with diverse teams and clients.

Responsibilities

• Design, implement, and optimize scalable Big Data ETL pipelines using Hive and Spark.
• Manage and maintain HDFS-based data storage systems.
• Collaborate with Data Engineers, DevOps, and Application teams for data flow orchestration and optimization.
• Perform root cause analysis and provide L2/L3 production support for data infrastructure issues.
• Monitor system performance and automate processes in a Linux-based environment.
• Ensure high availability and disaster recovery setup for big data platforms.
• Follow DevOps best practices for CI/CD pipelines, version control, and deployment.
• Document systems architecture, processes, and runbooks.

What This Job Offers

• Career Level: Senior
• Industry: Professional, Scientific, and Technical Services
• Number of Employees: 5,001-10,000 employees