Senior Data Engineer (Scala Development)

Techstra Solutions • Pittsburgh, PA

7d

About The Position

We are seeking an experienced Senior Data Engineer with strong Scala development expertise to build and support large-scale data processing solutions. This role will focus on developing high-performance data pipelines and applications using Spark and the Hadoop ecosystem, supporting critical enterprise data platforms. The ideal candidate brings strong problem-solving skills, a deep understanding of distributed data processing, and an engineering mindset that considers the full lifecycle of code—from design and development through deployment, performance optimization, and long-term production stability.

Requirements

• Bachelor’s degree in Computer Science, Information Technology, or related field.
• 8–10 years of experience in data engineering or software development roles.
• Strong Scala development experience (primary programming language).
• Hands-on experience with Apache Spark for distributed data processing.
• Experience working within the Hadoop ecosystem (HDFS, Hive, YARN, or related technologies).
• Basic to intermediate SQL skills for querying and manipulating data.
• Strong problem-solving and analytical skills.
• Demonstrated ability to design solutions while considering the entire software lifecycle, including performance, monitoring, and production stability.

Nice To Haves

• Experience working in Corporate & Investment Banking (C&IB) environments.
• Familiarity with large-scale enterprise data platforms handling high data volumes.
• Experience with data pipeline optimization and performance tuning.
• Exposure to modern data platforms, streaming frameworks, or cloud-based data services.
• Experience working in Agile development environments.

Responsibilities

• Design, develop, and maintain scalable data pipelines and distributed data processing applications using Scala and Spark.
• Build and optimize large-scale batch and data processing frameworks within the Hadoop ecosystem.
• Write efficient and maintainable Scala-based applications for enterprise data platforms.
• Develop and maintain SQL-based queries and transformations for data processing and validation.
• Collaborate with data architects, engineers, and business stakeholders to translate business requirements into scalable data solutions.
• Ensure data pipeline performance, reliability, and stability in production environments.
• Troubleshoot and resolve performance issues across distributed systems.
• Contribute to best practices around data engineering, code quality, testing, and operational support.
• Support production environments and continuously improve monitoring, resiliency, and stability of data systems.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level