Data Engineer - SQL

R+L Carriers•Ocala, FL

About The Position

We are seeking a Data Engineer with strong SQL skills to join our team. In this role, you will be responsible for developing, maintaining, monitoring, and optimizing data pipelines that move information from operational business systems into our enterprise data platform and data lake. You will design and maintain ETL and ELT processes using Azure Data Factory and related Azure data technologies, and manage our Azure Data Lake environment. This position involves architecting new data feeds, developing processes for ingesting various data formats, and troubleshooting data-related issues. You will also focus on improving data pipeline performance, developing SQL queries and curated datasets for reporting tools like Power BI, and designing data models that support enterprise reporting and analytics. A key aspect of this role is establishing data quality controls, standards for data management, and supporting data governance initiatives. You will document data sources, pipelines, and models, and participate in the evaluation of new data technologies. Collaboration with various teams, adherence to security practices, and proactive identification of improvements are essential.

Requirements

3+ years of experience in data engineering, data integration, database development, business intelligence engineering, or a related technical role.
Strong experience with Microsoft Azure data technologies, particularly Azure Data Factory, Azure Data Lake Storage, and DataBricks.
Strong SQL skills with experience developing complex queries, views, stored procedures, transformations, and reporting datasets.
Experience designing, developing, and maintaining ETL and ELT pipelines that integrate multiple enterprise data sources.
Experience integrating data through REST APIs, databases, file transfers, JSON, CSV, flat files, and other common integration methods.
Strong understanding of relational databases, data structures, database relationships, data types, indexing, and query optimization.
Understanding of data warehousing concepts, dimensional modeling, fact and dimension tables, star schemas, and data structures designed for analytics.
Experience preparing and modeling data for Power BI or similar business intelligence and visualization platforms.
Understanding of Power BI data requirements, semantic models, relationships, refresh processes, and reporting performance considerations.
Experience troubleshooting and optimizing data pipelines, SQL queries, transformations, and large data processing workloads.
Knowledge of incremental loading, change data capture concepts, data synchronization, data dependencies, and historical data management.
Experience implementing data validation, reconciliation, monitoring, logging, error handling, and data quality processes.
Understanding of data governance concepts including data ownership, business definitions, lineage, documentation, access controls, quality, and trusted data sources.
Ability to understand business processes and translate business reporting requirements into scalable technical data solutions.
Ability to analyze data discrepancies and determine whether problems originate within source systems, integrations, transformation logic, data models, or reporting layers.
Experience with Python for data processing, automation, integration, or data engineering is required.
Familiarity with Git or other source control and development lifecycle practices.
Familiarity with Azure security, role-based access control, service accounts, credentials, and secure data integration practices.
Strong analytical, troubleshooting, communication, organization, documentation, and problem-solving skills.
Ability to explain technical data concepts, issues, dependencies, and recommendations clearly to both technical and non-technical stakeholders.
Ability to work independently while collaborating cross-functionally with application development, infrastructure, analytics, project management, and business teams.
Ability to take ownership of critical data processes and proactively identify potential failures, performance issues, data quality concerns, and architectural improvements.
Ability to manage multiple priorities in a fast-paced environment with changing business and reporting needs.
Ability to read, write, and speak English fluently.

Nice To Haves

Experience with data originating from ERP, CRM, transportation, logistics, financial, warehouse, customer service, or other enterprise business systems is preferred.

Responsibilities

Develop, maintain, monitor, and optimize data pipelines that move information from operational business systems into the company’s enterprise data platform and data lake.
Design and maintain ETL and ELT processes using Azure Data Factory and related Azure data technologies.
Manage and support the company’s Azure Data Lake environment, including data organization, storage structures, processing workflows, access, and overall platform reliability.
Architect new data feeds and integrations from enterprise applications, APIs, databases, files, and other internal and external data sources.
Develop processes for ingesting structured and semi-structured data including SQL data, APIs, JSON, CSV, flat files, and other common data formats.
Troubleshoot data feed failures, pipeline errors, processing issues, data discrepancies, and other problems affecting the availability or accuracy of business data.
Monitor and improve data pipeline performance, processing times, query performance, resource utilization, and overall data platform efficiency.
Develop and maintain SQL queries, views, stored procedures, transformations, and reusable data structures used by reporting and business applications.
Develop curated datasets and reporting views that provide Power BI and other analytics tools with consistent, reliable, and understandable business data.
Work with Power BI developers and data analysts to understand reporting requirements and translate business needs into appropriate data structures and models.
Design and maintain data models that support enterprise reporting, analytics, operational reporting, historical analysis, and KPI measurement.
Apply dimensional modeling principles where appropriate, including fact tables, dimension tables, relationships, measures, historical data, and common business entities.
Improve business visibility into data by identifying opportunities to make information easier to access, understand, analyze, and use for decision-making.
Establish and maintain appropriate data quality controls, validation processes, reconciliation procedures, and monitoring to identify missing, inaccurate, duplicated, or inconsistent data.
Develop processes for incremental data loading, change detection, historical data retention, and efficient processing of large data sets.
Establish standards for data naming, definitions, structure, documentation, ownership, lineage, and appropriate use across enterprise reporting.
Support data governance initiatives that create consistent definitions and trusted sources for customers, employees, shipments, financial information, operational metrics, and other key business entities.
Help establish consistent enterprise KPIs and reporting definitions to reduce conflicting calculations and different versions of the same business metric.
Document data sources, pipelines, transformations, dependencies, business rules, data models, and reporting structures.
Maintain visibility into dependencies between source systems, data pipelines, transformations, reporting datasets, and downstream applications.
Participate in the evaluation and implementation of new data technologies, tools, integrations, and architectural improvements.
Work with application developers, infrastructure teams, data analysts, project managers, and business stakeholders to support new projects and data requirements.
Follow appropriate security, access control, data privacy, change management, testing, and development practices when working with enterprise data.
Proactively identify opportunities to improve data reliability, processing performance, automation, scalability, maintainability, and overall data architecture.