W2PE - Engineer

Trace3•Irvine, CA

15d•$120,000 - $150,000•Remote

About The Position

Trace3 is seeking a Fabric Data Engineer to accelerate the enterprise Data Modernization program by building and refactoring scalable, metadata-driven ETL pipelines in Microsoft Fabric. This role will design and implement medallion architecture (Bronze, Silver, Gold) using parameterized Python notebooks to move source data from landing through governed, trusted analytical layers, reducing dependence on legacy ETL platforms and enabling faster, more reliable speed-to-insight for enterprise reporting. The Fabric Data Engineer will work hands-on to modernize existing pipelines, apply reusable and parallelized data-processing patterns, and support the broader Fabric medallion implementation already underway as part of Phase 1C of the program. This is a delivery-focused engineering role with an explicit expectation of transferring architecture, code, and operating knowledge to the current data-engineering team as the program matures.

Requirements

3+ years of hands-on data engineering experience building production ETL/ELT pipelines.
Demonstrated experience designing and implementing metadata-driven data pipelines (parameterized, configuration-based pipeline patterns rather than one-off builds).
Strong proficiency in Python, including experience building notebook-based data-processing pipelines (PySpark or equivalent) for parallel/distributed processing.
Hands-on experience with Microsoft Fabric and/or Databricks, including notebooks, Lakehouse/Warehouse objects, and pipeline orchestration.
Practical experience implementing medallion (Bronze/Silver/Gold) data architecture, including data validation, deduplication, and historization patterns.
Working knowledge of SQL and relational data warehousing concepts.
Experience with cloud-based analytics platforms (Azure preferred).
Strong troubleshooting skills and ability to document and communicate technical designs to both technical and non-technical stakeholders.

Nice To Haves

Experience migrating pipelines from Azure Data Factory or similar legacy ETL platforms into Fabric-native services.
Familiarity with Power BI semantic models and downstream reporting consumption patterns.
Microsoft Fabric certifications (e.g., DP-600, DP-700) or equivalent Databricks certifications.
Experience operating in a consulting or client-delivery environment.

Responsibilities

Design, build, and maintain metadata-driven data pipelines in Microsoft Fabric that are reusable across source systems and entities rather than built one-off per pipeline.
Develop and maintain a metadata/configuration-driven framework that governs ingestion, transformation, and load behavior across the pipeline estate.
Standardize ingestion patterns using parameterized notebooks to minimize duplicate logic and simplify ongoing maintenance.
Build and refactor Bronze, Silver, and Gold layers in the Fabric Lakehouse/Warehouse, ensuring each layer meets defined validation and quality gates before promotion.
Preserve source fidelity in the landing zone and implement conformed, deduplicated, and historized data in Silver in support of downstream Gold transformations.
Apply warehouse-side transformation logic and support native-service patterns (e.g., Dataflows Gen2, Fabric Data Factory) as part of the broader platform modernization effort.
Develop Python notebooks (including PySpark-based processing) to support scalable, parallel data-processing pipelines that improve data-processing efficiency and reduce time to insight.
Build restartable, incremental, and schema-aware processing logic that scales across growing data volumes and source counts.
Implement data validation and quality checks at each layer (landing, Bronze, Silver, Gold) rather than treating validation as an end-stage activity.
Maintain operational evidence for pipeline runs, notebook executions, merges, and quality results to support troubleshooting and auditability.
Support performance tuning, refresh-frequency optimization, and capacity-efficient pipeline design within Fabric.
Partner with the Director, Data and Analytics Enablement and the current data-engineering team to align pipeline design with governance, metadata, and certified-dataset standards.
Document architecture, code, and runbooks, and actively transfer operating knowledge so the current data-engineering team can independently operate, troubleshoot, and extend delivered pipelines.
Collaborate with reporting, governance, and platform stakeholders to ensure pipeline outputs support trusted, consistent enterprise reporting.