Data Engineer (Snowflake/DataStage)

VSP Vision Care•Town of Owego, NY

1d•$63,000 - $108,675

About The Position

The Data Engineer creates and maintains data pipelines for key data and analytics capabilities in the enterprise. This position works in collaboration with analytics and data warehousing staff, DBAs and subject matter experts to create reliable processes that load targeted data with integrity and quality, enabling it for strategic use by the business. Collaborate within an agile, multi-disciplinary team to develop optimal data integration and transformation solutions Document and analyze data requirements (functional and non-functional) to develop scalable, automated, fault-tolerant data pipeline solutions for business and technology initiatives Profile data to assess the accuracy and completeness of data sources and work with the business partners to mitigate issues Build and maintain data pipelines for using appropriate tools and practices in development, test, and production environments. Design with modularity to leverage code reuse wherever possible Create data mappings, programs, routines, and SQL to acquire data from legacy, web, cloud, and purchased package environments into the analytics environment Use a mix of ELT, ETL, data virtualization, and other methods to optimize the balance of minimal data movement against performance Maintain metadata management processes and documentation Monitor data quality to detect emerging issues and consult with the team to create transformation rules to cleanse against defined rules and standards Participate in code reviews and unit testing to optimize performance and minimize issues Job Specifications Typically has the following skills or abilities: Bachelor’s degree in computer science, data science, statistics, economics, or related functional area; or equivalent experience Effective written and verbal communication skills with the ability to gather requirements and effectively collaborate with teammates and business partners 4+ years of experience working in a development team providing analytical capabilities 4+ years of hands-on experience in the data space, spanning data preparation, SQL, integration tools, ETL/ELT / data pipeline design SQL coding experience Familiarity with agile development environments (Scrum, Kanban) with a focus on Continuous Integration and Delivery Previous experience using a data integration platform (IBM InfoSphere DataStage, Oracle Data Integrator, Informatica PowerCenter, MS SSIS, AWS Glue, Denodo), and familiarity with data warehouse MPP platforms such Snowflake, Netezza, Teradata, Redshift, etc Familiarity with event store and stream processing (Apache Kafka and platforms like Confluent) and with API development and management platforms (MuleSoft, Axway) is also beneficial Capable of focusing on a specific set of tasks while also ensuring alignment to a broader strategic design Exhibits the traits of a pro-active, self-driven contributor, who values continual learning and the adoption of new technology

Requirements

Bachelor’s degree in computer science, data science, statistics, economics, or related functional area; or equivalent experience
Effective written and verbal communication skills with the ability to gather requirements and effectively collaborate with teammates and business partners
4+ years of experience working in a development team providing analytical capabilities
4+ years of hands-on experience in the data space, spanning data preparation, SQL, integration tools, ETL/ELT / data pipeline design
SQL coding experience
Familiarity with agile development environments (Scrum, Kanban) with a focus on Continuous Integration and Delivery
Previous experience using a data integration platform (IBM InfoSphere DataStage, Oracle Data Integrator, Informatica PowerCenter, MS SSIS, AWS Glue, Denodo), and familiarity with data warehouse MPP platforms such Snowflake, Netezza, Teradata, Redshift, etc
Familiarity with event store and stream processing (Apache Kafka and platforms like Confluent) and with API development and management platforms (MuleSoft, Axway) is also beneficial
Capable of focusing on a specific set of tasks while also ensuring alignment to a broader strategic design
Exhibits the traits of a pro-active, self-driven contributor, who values continual learning and the adoption of new technology

Nice To Haves

Strong experience in Snowflake, IBM DataStage, and SQL to design, build, and optimize scalable data solutions
Experience in data warehousing, analytics enablement, and high-performance data pipelines using modern orchestration and security tools such as Apache Airflow and Vault speed

Responsibilities

Collaborate within an agile, multi-disciplinary team to develop optimal data integration and transformation solutions
Document and analyze data requirements (functional and non-functional) to develop scalable, automated, fault-tolerant data pipeline solutions for business and technology initiatives
Profile data to assess the accuracy and completeness of data sources and work with the business partners to mitigate issues
Build and maintain data pipelines for using appropriate tools and practices in development, test, and production environments.
Design with modularity to leverage code reuse wherever possible
Create data mappings, programs, routines, and SQL to acquire data from legacy, web, cloud, and purchased package environments into the analytics environment
Use a mix of ELT, ETL, data virtualization, and other methods to optimize the balance of minimal data movement against performance
Maintain metadata management processes and documentation
Monitor data quality to detect emerging issues and consult with the team to create transformation rules to cleanse against defined rules and standards
Participate in code reviews and unit testing to optimize performance and minimize issues