Forward Deployed - Data Architect

ShyftLabs

About The Position

ShyftLabs is seeking an experienced, client-facing Data Architect to provide architectural leadership and hands-on technical expertise across enterprise data engineering initiatives. The role combines solution architecture, technical consulting, and delivery oversight, translating complex business requirements into scalable, secure, and reliable data solutions. The architect will collaborate with client stakeholders, engineering teams, and technology partners to set architectural direction, establish engineering standards, and guide data platform implementations from design through production. The role includes designing modern data platforms with Azure Databricks and Delta Lake; leading discovery, workshops, and architecture reviews; and providing hands-on guidance using SQL, Python/PySpark, Apache Airflow, and dbt Core. The architect will define ingestion patterns, implement Medallion Architecture, establish data governance and modeling practices, oversee technical quality, and guide CI/CD, reliability, performance, cost optimization, and AI-assisted development practices. The position also coordinates distributed engineering teams, communicates decisions and risks, and maintains architectural ownership through delivery and ongoing optimization.

Requirements

At least 8 years of overall experience in data engineering, data architecture, or related technical roles, with hands-on experience designing and delivering production-grade data platforms.
Proven experience in a client-facing technical role, such as Data Architect, Solutions Architect, Technical Consultant, or Lead Data Engineer, working directly with enterprise customers or stakeholders.
Strong hands-on expertise with Azure Databricks, including designing and implementing scalable enterprise data solutions.
Deep understanding of Delta Lake, including data management, processing patterns, and performance optimization.
Experience with Unity Catalog, including data governance, access controls, lineage, and enterprise data management.
Strong proficiency in SQL and Python/PySpark for data processing, transformation, and pipeline development.
Hands-on experience with Apache Airflow for orchestration and dbt Core for data transformation, testing, and modeling.
Experience designing ingestion pipelines across APIs, relational databases, files, and other enterprise data sources.
Strong understanding of Medallion Architecture, data products, semantic modeling, and data quality frameworks.
Experience with CI/CD, Git, automated testing, monitoring, and production data pipeline management.
Ability to review and challenge technical architectures, code, data models, and engineering deliverables and provide constructive, actionable feedback.
Experience translating business requirements into technical specifications, architectural designs, and implementation roadmaps.
Ability to evaluate technical trade-offs and communicate recommendations to technical teams, business stakeholders, and client leadership.
Experience working with external technology partners, third-party delivery teams, or distributed engineering organizations.
Strong communication, presentation, stakeholder management, and problem-solving skills.
Ability to manage competing priorities, navigate ambiguity, and build trusted client relationships while maintaining technical credibility.

Nice To Haves

Familiarity with Databricks-native ingestion technologies, including Lakeflow Connect and Auto Loader.
Familiarity with AI-assisted development tools and spec-driven development practices.

Responsibilities

Lead client-facing architecture discussions to understand business objectives, technical requirements, existing data environments, and architectural challenges.
Design and recommend modern data platform architectures using Azure Databricks, Delta Lake, and cloud-based data engineering technologies.
Facilitate technical workshops, discovery sessions, and architecture reviews to assess current environments, identify gaps, evaluate options, and define target-state architectures.
Develop, validate, and review data engineering solutions using SQL, Python/PySpark, Apache Airflow, and dbt Core.
Establish standards for ingestion pipelines integrating APIs, databases, files, and cloud data sources, including Databricks Lakeflow Connect and Auto Loader.
Define reusable architectural patterns, implementation guidelines, and engineering best practices across projects and distributed delivery teams.
Design and guide implementation of Bronze, Silver, and Gold Medallion Architecture layers.
Establish practices for Unity Catalog, data access controls, lineage, semantic modeling, data discoverability, and enterprise data governance.
Review architecture designs, code, data models, and pipelines for requirements alignment, performance, security, and maintainability.
Define standards for data validation, automated testing, pipeline monitoring, error handling, and operational readiness.
Guide Git-based workflows, automated testing, CI/CD pipelines, and controlled deployments across development, testing, and production environments.
Evaluate platform performance, processing efficiency, scalability, and cloud consumption, and recommend optimizations.
Encourage effective AI-assisted development tools and specification-driven engineering while maintaining architectural integrity, code quality, and delivery standards.
Provide technical direction to distributed engineering teams and facilitate collaboration among clients, internal engineers, and external technology partners.
Communicate architectural decisions, trade-offs, dependencies, risks, and recommendations to technical and non-technical stakeholders.
Maintain architectural ownership from discovery and solution design through implementation, deployment, and ongoing optimization.