Big Data Engineer

Select Minds•Lansing, MI

1d•Onsite

About The Position

We are seeking an experienced Big Data Engineer to design, develop, and optimize enterprise-scale big data solutions. The ideal candidate will have strong hands-on expertise in the Hadoop ecosystem, Spark, Scala, Hive, Apache NiFi, and Kafka, along with experience building scalable data pipelines and supporting modern CI/CD practices. Candidates should have worked directly with clients, participated in requirement gathering sessions, and delivered end-to-end Big Data implementations.

Requirements

8+ years of experience in Big Data technologies.
Strong hands-on experience with Spark, Scala, Hive (HQL), and the Hadoop ecosystem.
Experience implementing Apache NiFi and Apache Kafka.
Proficiency in Shell Scripting or Python.
Experience with PostgreSQL.
Familiarity with scheduling tools such as Oozie and Control-M.
Working knowledge of Git, Jenkins, Nexus, Maven, Ansible, and SonarQube.
Strong understanding of CI/CD practices.
Experience in client-facing roles, including requirement gathering workshops.
Minimum of two end-to-end Big Data implementation projects.
Ability to work independently and contribute effectively within Agile teams.

Responsibilities

Design, develop, and optimize large-scale data processing pipelines using Spark, Scala, and Hive.
Build and maintain data ingestion workflows using Apache NiFi.
Develop and support real-time data streaming solutions using Apache Kafka.
Monitor application logs and system health using Kibana, Elasticsearch, and Logstash.
Implement and support CI/CD pipelines using Git, Jenkins, and Nexus.
Ensure data quality, integrity, and governance across Big Data platforms.
Participate in architecture discussions and recommend scalable design solutions.
Troubleshoot and resolve complex production issues across the Big Data stack.
Collaborate with cross-functional teams in an Agile environment.
Mentor junior engineers and contribute to technical best practices.