Senior Fabric Data Engineer

Spirit Omega

3d•Remote

About The Position

We are seeking a Senior Fabric Data Engineer with strong hands-on Microsoft Fabric experience to support migration and modernization of learning-platform data from Saba and Cura into the Cornerstone Galaxy ecosystem. This role will focus on building scalable ingestion, transformation, business-rule, and curated data layers to enable analytics, compliance reporting, skills insights, and downstream integrations.

Requirements

7+ years of experience in data engineering and enterprise data integration.
Strong hands-on experience with Microsoft Fabric and related Azure data services.
Strong experience with PySpark, SparkSQL, Python, and SQL/T-SQL.
Experience building ingestion and transformation pipelines from APIs, files, databases, and third-party platforms.
Experience with curated data layers, data modeling, and analytics-ready datasets.
Experience tuning data pipelines for scale, resiliency, and cost optimization.
Working knowledge of security, access controls, and data governance practices.
Experience with Azure DevOps and Git-based deployment automation.
Strong analytical, troubleshooting, and communication skills.

Nice To Haves

Experience with learning, HR, or talent data domains.
Familiarity with ADF, Synapse, Databricks, Azure Functions, or Logic Apps.
Exposure to AI tools such as GitHub Copilot and Claude to accelerate engineering productivity and solution design.
Experience with data governance tooling such as Purview.
Exposure to AI/ML-ready data preparation and feature engineering practices.

Responsibilities

Design and implement robust data solutions using Microsoft Fabric, including OneLake, Fabric Pipelines, Lakehouse, Warehouse, and Notebooks.
Build and optimize scalable ingestion and transformation pipelines using PySpark, Python, and SQL across structured and unstructured datasets.
Migrate and harmonize data from legacy source systems including Saba, Cura, and related enterprise platforms.
Develop API-driven and scheduled workflows to ingest data from internal and external systems and support downstream processing.
Implement business rules for learning history, completion status, certifications, compliance, skills, and learner profile data.
Build curated and consumption-ready datasets/models for analytics, dashboards, and operational reporting.
Perform SQL development and tuning for reporting, analytics, and downstream consumption.
Implement automated data quality checks, reconciliations, monitoring, exception handling, and audit logging.
Enforce secure access controls using Azure AD, RBAC, managed identities, and secure connectivity patterns.
Support CI/CD, code promotion, release management, and operational monitoring in Agile delivery.