Lead Data Integration Engineer
About The Position
Lead Data Integration Engineer positions offered by Kimberly-Clark USA, LLC (Irving, Texas). Work with Technical Architects, Product Owners and Business teams to translate requirements into technical design for data modelling and data integration. Utilize background in data warehousing, data modelling and ETL/ELT data processing patterns. Design and develop ETL/ELT pipelines with reusable patterns and frameworks. Design and build efficient SQLs to process and curate the data sets in HANA, Azure and Snowflake. Design and review data ingestion frameworks leveraging Python, Spark, Azure Data Factory, Snowpipe, etc. Design and build Data Quality models and ABCR frameworks to ingest, validate, curate and prepare the data for consumption. Review functional domain and business needs and identify the gaps in the requirements proactively prior to implementing solutions. Work with platform teams to design and build processes for automation in pipeline build, testing and code migrations.
Requirements
- Requires a bachelor’s or foreign equivalent degree in computer science, applied computer science, IT or a related field and 10 years of experience in the job offered or 10 years of experience designing, developing, and building ETL/ELT pipelines, procedures, and SQLs on MPP platforms including HANA, Snowflake, Greenplum, or Teradata.
- Also requires 5 years of experience: designing and building metadata driven data ingestion frameworks, building SAP BO/Data Services, Azure Data Factory, SnowSQL, and Snowpipe, and building mini-batch, real-time and event-driven data processing jobs; designing and developing object stores (including Azure ADLS, HDFS, and GCP Cloud Storage), Row and Columnar databases (including Azure SQL DW, SQL Server, Snowflake, Teradata, PostgresSQL, and Oracle), NoSQL databases (including CosmosDB, MongoDB, and Cassandra), ElasticSearch, Redis, and Data processing platforms Spark, Databricks, and SnowSQL; and leveraging Azure Stream Analytics, Azure Analysis Services, Data Lake Analytics, HDInsight, HDP, Spark, Databricks, MapReduce, Pig, Hive, Tez, SSAS, Watson Analytics, and SPSS for Data Analytics development and solution architecture.
Responsibilities
- Work with Technical Architects, Product Owners and Business teams to translate requirements into technical design for data modelling and data integration.
- Utilize background in data warehousing, data modelling and ETL/ELT data processing patterns.
- Design and develop ETL/ELT pipelines with reusable patterns and frameworks.
- Design and build efficient SQLs to process and curate the data sets in HANA, Azure and Snowflake.
- Design and review data ingestion frameworks leveraging Python, Spark, Azure Data Factory, Snowpipe, etc.
- Design and build Data Quality models and ABCR frameworks to ingest, validate, curate and prepare the data for consumption.
- Review functional domain and business needs and identify the gaps in the requirements proactively prior to implementing solutions.
- Work with platform teams to design and build processes for automation in pipeline build, testing and code migrations.
Benefits
- Eligibility for a target bonus
- 401(k) and Profit Sharing plan
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
