Senior Data Engineer (Python / PySpark / Kafka)

Computer Task Group, IncPiscataway Township, NJ
8d$70 - $80

About The Position

CTG is seeking to fill a Senior Data Engineer (Python / PySpark / Kafka) position for our client. Location: Piscataway, NJ Duration: 12 months Duties: Design and develop scalable data pipelines and data processing systems for enterprise analytics. Build distributed data workflows using Python and PySpark. Develop real-time data streaming solutions using Apache Kafka. Integrate data from multiple sources and ensure data quality, reliability, and performance. Collaborate with data scientists, analysts, and engineering teams to support analytics and machine learning initiatives. Monitor, troubleshoot, and optimize large-scale data processing environments. Skills: Strong hands-on experience with Python. Extensive experience with PySpark for distributed data processing. Proven experience with Kafka and real-time/stream processing architectures. Experience building ETL/ELT pipelines and large-scale data platforms. Strong problem-solving, performance tuning, and data engineering best practices. Experience: 8+ years of experience in Data Engineering or Big Data development. Demonstrated experience working with Python, PySpark, and Kafka in production environments. Experience with large-scale distributed data systems and modern data architectures. Education: Bachelor’s degree in Computer Science, Information Systems, Engineering, or related field, or equivalent experience. Excellent verbal and written English communication skills and the ability to interact professionally with a diverse group are required. CTG does not accept unsolicited resumes from headhunters, recruitment agencies, or fee based recruitment services for this role. To Apply: To be considered, please apply directly to this requisition using the link provided. For additional information, please contact Laura Dominguez at [email protected]. Kindly forward this to any other interested parties. Thank you! The expected base salary for this position ranges from $70.00 to $80.00/hour. Salary offers are based on a wide range of factors including relevant skills, training, experience, education, market factors, and where applicable, licensure or certifications obtained. In addition to salary, a competitive benefit package is also offered. About CTG CTG, a Cegeka company, is at the forefront of digital transformation, providing IT and business solutions that accelerate project momentum and deliver desired value. Over nearly 60 years, we have earned a reputation as a faster and more reliable, results-driven partner. Our vision is to be an indispensable partner to our clients and the preferred career destination for digital and technology experts. CTG leverages the expertise of over 9,000 team members in 19 countries to provide innovative solutions. Together, we operate across the Americas, Europe, and India, working in close cooperation with over 3,000 clients in many of today's highest-growth industries. For more information, visit www.ctg.com. Our culture is a direct result of the people who work at CTG, the values we hold, and the actions we take. In other words, our people define our culture. It's a living, breathing thing that is renewed every day through the ways we engage with each other, our clients, and our communities. Part of our mission is to cultivate a workplace that attracts and develops the best people. CTG will consider for employment all qualified applicants including those with criminal histories in a manner consistent with the requirements of all applicable local, state, and federal laws. CTG is an Equal Opportunity Employer. CTG will assure equal opportunity and consideration to all applicants and employees in recruitment, selection, placement, training, benefits, compensation, promotion, transfer, and release of individuals without regard to race, creed, religion, color, national origin, sex, sexual orientation, gender identity and gender expression, age, disability, marital or veteran status, citizenship status, or any other discriminatory factors as required by law. CTG is fully committed to promoting employment opportunities for members of protected classes.

Requirements

  • Strong hands-on experience with Python.
  • Extensive experience with PySpark for distributed data processing.
  • Proven experience with Kafka and real-time/stream processing architectures.
  • Experience building ETL/ELT pipelines and large-scale data platforms.
  • Strong problem-solving, performance tuning, and data engineering best practices.
  • 8+ years of experience in Data Engineering or Big Data development.
  • Demonstrated experience working with Python, PySpark, and Kafka in production environments.
  • Experience with large-scale distributed data systems and modern data architectures.
  • Bachelor’s degree in Computer Science, Information Systems, Engineering, or related field, or equivalent experience.
  • Excellent verbal and written English communication skills and the ability to interact professionally with a diverse group are required.

Responsibilities

  • Design and develop scalable data pipelines and data processing systems for enterprise analytics.
  • Build distributed data workflows using Python and PySpark.
  • Develop real-time data streaming solutions using Apache Kafka.
  • Integrate data from multiple sources and ensure data quality, reliability, and performance.
  • Collaborate with data scientists, analysts, and engineering teams to support analytics and machine learning initiatives.
  • Monitor, troubleshoot, and optimize large-scale data processing environments.
© 2024 Teal Labs, Inc
Privacy PolicyTerms of Service