Data Engineer - Microsoft Fabric / Azure - Intermediate

Innosystech•Philadelphia, PA

2d•Remote

About The Position

Requiring a senior engineer who can work independently while collaborating with data architects, analysts, application teams, and business stakeholders. This role involves designing, developing, and maintaining scalable enterprise data pipelines using Microsoft Fabric and Azure data services. The engineer will build ingestion frameworks for various data sources, develop complex data transformation workflows, and create data layers optimized for analytical consumption. Responsibilities also include implementing data quality checks, troubleshooting pipeline issues, collaborating on technical solutions, supporting production deployments, and documenting data flows and designs.

Requirements

7+ years of experience in data engineering and enterprise data integration.
Strong hands-on experience with Microsoft Fabric and related Azure data services.
Strong experience with PySpark, SparkSQL, Python, and SQL/T-SQL.
Experience building ingestion and transformation pipelines from APIs, files, databases, and third-party platforms.
Experience with curated data layers, data modeling, and analytics-ready datasets.
Experience tuning data pipelines for scale, resiliency, and cost optimization.
Working knowledge of security, access controls, and data governance practices.
Experience with Azure DevOps and Git-based deployment automation.
Strong understanding of ETL/ELT architecture and data integration patterns.
Experience implementing data-quality checks, error handling, monitoring, and recovery mechanisms.
Strong troubleshooting and analytical skills.
Ability to work effectively in a distributed, fully remote engineering environment.

Nice To Haves

Experience with Microsoft Fabric Lakehouse and Warehouse architectures.
Experience with OneLake.
Experience with Fabric Data Factory / Data Pipelines.
Experience with Azure Data Factory.
Experience with Azure Data Lake Storage (ADLS Gen2).
Experience with Delta Lake / Delta tables.
Knowledge of medallion architecture – Bronze, Silver, and Gold data layers.
Experience supporting enterprise analytics or reporting platforms such as Power BI.
Experience designing reusable ingestion frameworks and metadata-driven pipelines.
Understanding of data observability, lineage, metadata management, and governance frameworks.
Experience working within large enterprise or SaaS/product technology environments.

Responsibilities

Design, develop, and maintain scalable enterprise data pipelines using Microsoft Fabric and Azure data services.
Build ingestion frameworks for structured and semi-structured data from REST APIs, flat files, relational databases, cloud platforms, third-party applications and enterprise systems.
Develop complex data transformation and processing workflows using PySpark, SparkSQL, Python, SQL, and T-SQL.
Build and maintain raw, curated, and analytics-ready data layers supporting reporting, analytics, and downstream applications.
Design efficient data models and datasets optimized for analytical consumption.
Implement data quality, validation, reconciliation, and monitoring mechanisms across ingestion and transformation pipelines.
Troubleshoot pipeline failures, performance bottlenecks, data-quality issues, and integration problems.
Collaborate with architects and business teams to translate data requirements into scalable technical solutions.
Support production deployments, monitoring, troubleshooting, and continuous improvement of the data platform.
Document data flows, transformation logic, technical designs, dependencies, and operational procedures.