Data & Integration Engineer
About The Position
The Data and Integration Engineer is responsible for designing, developing, and maintaining data and integration solutions that enable seamless data flow across enterprise systems. This role requires expertise in ETL/ELT processes, API integrations, data warehousing, and cloud-based data platforms. The ideal candidate will work closely with business stakeholders, developers, and data teams to ensure efficient data management, high-quality data pipelines, and scalable integration solutions. ESSENTIAL DUTIES AND RESPONSIBILITIES Responsibilities: Design, develop, and maintain ETL/ELT pipelines to extract, transform, and load data from multiple sources into data lakes and data warehouses. Implement data transformation logic to ensure data quality, consistency, and performance. Automate data ingestion and transformation processes using tools like Azure Data Factory (ADF), SSIS, or AWS Glue. Work with RESTful and SOAP APIs, JSON, XML, and other data formats. Develop and manage API-based integrations between enterprise applications, cloud platforms, and third-party services. Implement authentication and security protocols (OAuth, JWT, API keys). Design and optimize database structures for data warehousing and reporting solutions. Write complex SQL queries, stored procedures, and views to support data extraction and analytics. Work with relational databases (SQL Server, PostgreSQL, Snowflake) and NoSQL solutions. Develop and deploy data solutions in cloud environments like Azure, AWS, or Google Cloud. Utilize cloud storage, compute services, and serverless technologies for scalable data processing. Work with modern data lake and Lakehouse architectures. Implement data governance best practices, including metadata management, data lineage tracking, and access controls. Ensure compliance with data privacy regulations (GDPR, HIPAA, CCPA) and security best practices. Monitor data integration jobs and troubleshoot failures or performance issues. Create technical documentation, process workflows, and integration diagrams. Support cross-functional teams in identifying and resolving data-related issues. Knowledge, Skills, and Abilities: Experience in ETL/ELT tools (Azure Data Factory, SSIS, Informatica, Talend, etc.). Experience with SQL and database technologies (SQL Server, Snowflake, PostgreSQL, MySQL). Hands-on experience with APIs, web services, and integration tools. Familiarity with cloud data platforms (Azure Synapse, AWS Redshift, Google BigQuery). Experience with scripting languages (Python, PowerShell, or Bash) for automation. Knowledge of DevOps practices, CI/CD pipelines, and Infrastructure as Code (Terraform, ARM templates). Strong problem-solving and analytical skills. Excellent communication and collaboration abilities. Ability to work in a fast-paced environment and manage multiple priorities. Strong attention to detail and data accuracy.
Requirements
- Bachelor’s degree in Computer Science, Information Systems, or a related field
- 3+ years of experience in data engineering, data integration, or related roles
- Experience in ETL/ELT tools (Azure Data Factory, SSIS, Informatica, Talend, etc.).
- Experience with SQL and database technologies (SQL Server, Snowflake, PostgreSQL, MySQL).
- Hands-on experience with APIs, web services, and integration tools.
- Familiarity with cloud data platforms (Azure Synapse, AWS Redshift, Google BigQuery).
- Experience with scripting languages (Python, PowerShell, or Bash) for automation.
- Knowledge of DevOps practices, CI/CD pipelines, and Infrastructure as Code (Terraform, ARM templates).
- Strong problem-solving and analytical skills.
- Excellent communication and collaboration abilities.
- Ability to work in a fast-paced environment and manage multiple priorities.
- Strong attention to detail and data accuracy.
Responsibilities
- Design, develop, and maintain ETL/ELT pipelines to extract, transform, and load data from multiple sources into data lakes and data warehouses.
- Implement data transformation logic to ensure data quality, consistency, and performance.
- Automate data ingestion and transformation processes using tools like Azure Data Factory (ADF), SSIS, or AWS Glue.
- Work with RESTful and SOAP APIs, JSON, XML, and other data formats.
- Develop and manage API-based integrations between enterprise applications, cloud platforms, and third-party services.
- Implement authentication and security protocols (OAuth, JWT, API keys).
- Design and optimize database structures for data warehousing and reporting solutions.
- Write complex SQL queries, stored procedures, and views to support data extraction and analytics.
- Work with relational databases (SQL Server, PostgreSQL, Snowflake) and NoSQL solutions.
- Develop and deploy data solutions in cloud environments like Azure, AWS, or Google Cloud.
- Utilize cloud storage, compute services, and serverless technologies for scalable data processing.
- Work with modern data lake and Lakehouse architectures.
- Implement data governance best practices, including metadata management, data lineage tracking, and access controls.
- Ensure compliance with data privacy regulations (GDPR, HIPAA, CCPA) and security best practices.
- Monitor data integration jobs and troubleshoot failures or performance issues.
- Create technical documentation, process workflows, and integration diagrams.
- Support cross-functional teams in identifying and resolving data-related issues.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Entry Level
