Data Engineer

NDT Global Group•Calgary, AB

60d

About The Position

The Data Engineer will design, build, and maintain scalable data pipelines ensuring that data flows smoothly from its source to its destination efficiently and securely. The Data Engineer is also responsible for automating processes and tasks. This may include creating scripts to move files from one location to another or developing algorithms to process data more efficiently.

Requirements

Bachelor's degree in Computer Science, Data Engineering, or a related field.
Must be proficient in spoken and written English.
Minimum of 4+ years of experience working within a Data Engineering team building data automation solutions.
Strong proficiency in SQL and experience with relational and NoSQL databases.
Hands-on experience with ETL tools and frameworks such as SSIS or Azure Data Factory.
Familiarity with data warehousing concepts and tools such as Snowflake or BigQuery.
Knowledge of Python or other scripting languages used for data processing.
Experience working with cloud services such as Azure or AWS.
Understanding of data governance, security, and compliance best practices.
Strong problem-solving and communication skills.

Nice To Haves

Exposure to machine learning pipelines and advanced analytics environments is preferred.

Responsibilities

Design and build data pipelines by creating systems to collect, move, and transform data from various sources into a usable format.
Manage data storage by constructing and maintaining databases, data warehouses, and data lakes to store and organize data securely and efficiently.
Automate processes by building data automation through APIs that move data seamlessly across systems.
Collaborate with teams including Data Architects, BI teams, and other stakeholders to understand their data needs and build the necessary infrastructure.
Ensure data quality by implementing Master Data and related data processes to clean, validate, and monitor reliability and performance.
Maintain infrastructure by ensuring the data infrastructure is scalable, secure, and compliant with company policies. Monitor and troubleshoot data workflows to ensure high availability and performance. Develop and maintain documentation for data processes and architecture.