Data Engineer 2

IQVIA•Parsippany, NJ

1d•$135,000 - $155,000•Remote

About The Position

Design and develop software solutions requiring general domain knowledge and specialized business experience. Implement the components of the architecture in conjunction with distributed business and technical resources as well as external vendors. Analyze user requirements/needs and make decisions within limited parameters under regular supervision. Use experience in designing and building large-scale enterprise data solutions to meet business needs. Analyze large data sets to provide recommendations on how the data can be processed. Work at developing and maintaining applications that process, analyze, and report on large data sets. Implement Big Data pipelines using the AWS big data technologies (AWS EMR, AWS Glue, etc). Implement and review ETL/ELT jobs written in Scala using Apache Spark. Support and often lead projects within own organization with limited complexity, including the development of medium size system components. Review complex functional specifications and other validation deliverables as assigned. Implement Automated Unit tests and support QA in their implementation of Automated Acceptance tests. Debug issues identified in the existing pipeline code and implement fixes. Locate, analyze, and document performance issues in pipelines running in cloud environments, with the ability to propose and implement remedies. Provide peer support and help on-board new Software Engineers to the team. Collaborate with Product Owner, Scrum Master, DevOps, developers and testers in an Agile team environment with strong leadership, respect and cultivating of IQVIA Digital's agile culture. Evaluate emerging technologies to identify opportunities, trends and best practices that can be used to strengthen IQVIA Digital's technology platform and development practices.

Requirements

at least a bachelor’s degree in Computer Science, Management Information Systems or related field or foreign equivalent.
three (3) years’ experience in developing data applications with Scala Spark
three (3) years with utilizing Scala Spark, PySpark, Hive, or Airflow
three (3) years with utilizing AWS big data technologies (AWS EMR and AWS Glue)
three (3) years with writing clean and testable software code utilizing Scala, SQL and Terraform programming languages
three (3) years with implementing automated unit tests
three (3) years with working in an Agile environment
three (3) years with designing small to medium system components

Responsibilities

Design and develop software solutions requiring general domain knowledge and specialized business experience.
Implement the components of the architecture in conjunction with distributed business and technical resources as well as external vendors.
Analyze user requirements/needs and make decisions within limited parameters under regular supervision.
Use experience in designing and building large-scale enterprise data solutions to meet business needs.
Analyze large data sets to provide recommendations on how the data can be processed.
Work at developing and maintaining applications that process, analyze, and report on large data sets.
Implement Big Data pipelines using the AWS big data technologies (AWS EMR, AWS Glue, etc).
Implement and review ETL/ELT jobs written in Scala using Apache Spark.
Support and often lead projects within own organization with limited complexity, including the development of medium size system components.
Review complex functional specifications and other validation deliverables as assigned.
Implement Automated Unit tests and support QA in their implementation of Automated Acceptance tests.
Debug issues identified in the existing pipeline code and implement fixes.
Locate, analyze, and document performance issues in pipelines running in cloud environments, with the ability to propose and implement remedies.
Provide peer support and help on-board new Software Engineers to the team.
Collaborate with Product Owner, Scrum Master, DevOps, developers and testers in an Agile team environment with strong leadership, respect and cultivating of IQVIA Digital's agile culture.
Evaluate emerging technologies to identify opportunities, trends and best practices that can be used to strengthen IQVIA Digital's technology platform and development practices.