Lead Data Consultant (Microsoft Fabric)

Trility Consulting

2d•Remote

About The Position

Trility Consulting is seeking an experienced Senior Data Consultant for a client engagement modernizing an existing Azure Synapse data platform and migrating prioritized workloads to Microsoft Fabric. The role partners with Trility consultants and client engineers to refine Fabric architecture, build reusable metadata-driven ingestion frameworks, migrate data workloads into a lakehouse environment, develop scalable pipelines, validate migration accuracy, and establish automated CI/CD workflows from development through production. The engagement uses a collaborative Coach-and-Contribute delivery model. The consultant works hands-on with client engineers, contributes to architectural decisions, and establishes repeatable patterns that enable the team to maintain and extend the platform independently. Experience using Claude Code in an AI-assisted development workflow is essential. This is a remote W2 position. Trility can currently hire only W2 employees residing in Alabama, Colorado, Delaware, Florida, Georgia, Iowa, Kansas, Kentucky, Louisiana, Minnesota, Missouri, Montana, Nebraska, Nevada, North Carolina, Ohio, South Carolina, Tennessee, Texas, Virginia, or Wisconsin. The role requires U.S. residence, authorization to work in the United States without current or future sponsorship, and the ability to pass a background check.

Requirements

At least 5 years of experience working with Azure Data Services, including Microsoft Fabric, Azure Data Factory, Databricks, or Synapse.
Hands-on Microsoft Fabric experience in production environments, including lakehouses, pipelines, notebooks, medallion architecture, workspaces, and capacity management.
Strong experience building and maintaining production ETL/ELT pipelines and reusable data ingestion frameworks.
Demonstrated experience designing and implementing a metadata-driven ingestion framework, including configuration-based onboarding and extensible processing patterns.
Experience developing Spark notebooks and data transformation workflows using PySpark and Spark SQL.
Strong SQL skills for data transformation, validation, and performance tuning, including the ability to read and understand existing T-SQL logic.
Experience working with Azure Synapse, including analyzing existing pipelines and understanding complex data-processing dependencies.
Experience implementing medallion architecture patterns (Bronze, Silver, Gold).
Experience designing and implementing Git-based CI/CD workflows for data platform artifacts using GitHub Actions, Azure DevOps, or similar tooling.
Hands-on experience using Claude Code for AI-assisted software development, including directing code generation, working across multiple files, and critically reviewing generated code.
Experience validating data migrations and ensuring data quality, accuracy, and consistency across source and target systems.
Experience integrating data from REST APIs, relational databases, and structured or semi-structured data sources.
Familiarity with data governance, lineage, metadata management, and data catalog concepts.
Ability to collaborate effectively with technical teams, challenge existing approaches constructively, and participate in client-facing technical discussions.
Strong written and verbal communication skills.
Must reside within the United States.
Must be authorized to work in the United States without sponsorship now or in the future.
Must be able to pass a background check.

Nice To Haves

Experience migrating data platforms or workloads from Azure Synapse to Microsoft Fabric.
Experience working with Oracle and DB2 data sources.
Experience implementing incremental ingestion, change data capture (CDC), and source-specific loading strategies.
Experience developing data reconciliation and validation frameworks for large-scale migrations.
Experience evaluating Microsoft Purview or other data catalog and lineage solutions.
Experience working within consulting or professional services environments.

Responsibilities

Evaluate and refine the existing Microsoft Fabric architecture, including metadata structures, table-level event triggering, and reusable ingestion patterns.
Design and implement a metadata-driven ingestion framework that supports onboarding tables and sources through configuration rather than custom development.
Migrate prioritized data sources and workloads from Azure Synapse into Microsoft Fabric.
Develop and orchestrate Fabric data pipelines and Spark notebooks using PySpark and Spark SQL.
Analyze existing Azure Synapse pipelines, including complex and nested workflows, and translate dependencies and processing logic into Fabric solutions.
Build data validation and reconciliation processes to verify migration accuracy, including duplicate detection, key alignment, and record-level comparisons between Synapse and Fabric.
Analyze source-specific loading requirements and recommend full-load, incremental, or change data capture (CDC) patterns.
Implement Git-based CI/CD and automated environment promotion for Fabric artifacts, including pipelines and notebooks, using GitHub Actions, Azure DevOps, or similar tooling.
Use Claude Code to define implementation specifications, accelerate development, and review AI-generated code for quality, correctness, and maintainability.
Evaluate data dictionary, metadata, and lineage requirements, including potential catalog solutions such as Microsoft Purview.
Collaborate with client engineers through paired development, code reviews, and knowledge-sharing sessions to establish sustainable engineering standards.
Apply data engineering best practices for performance, testing, monitoring, documentation, security, and operational support.