16342 - Data Warehouse Architect 3

Smith Johnson Tech•Lansing, MI

1d•Hybrid

About The Position

The Data Engineering Lead serves as the primary State of Michigan technology lead, engaging with the University of Michigan proof-of-concept team to support adoption of best practices in the state hosting environment and train DAV data engineers. The role focuses on data modeling, Lakehouse and Medallion Architecture, distributed data architectures, and delivery of data and machine-learning products using an agile DevOps framework. It requires proficiency with data engineering, cloud, orchestration, and analytics technologies, including Spark, Kafka, Kubernetes, SageMaker, DBT, Databricks, SQL, and Power BI or Tableau. The position oversees data pipelines and data infrastructure, and establishes practices for data quality, reliability, security, governance, and data sharing. It also provides technical leadership and mentorship to data engineering staff and collaborates with data scientists, analysts, product teams, and business stakeholders. The work model is hybrid, with two days per week onsite. Relocation is allowed, but interviews must include an in-person round; occasional travel to Ann Arbor is expected.

Requirements

At least 4 years of hands-on experience hydrating an enterprise data lake on AWS, Azure, or Google Cloud from relational, semi-structured, and unstructured sources using extract-and-load pipelines.
At least 4 years of hands-on experience transforming cloud data-lake datasets into data products with traceability to critical data elements in systems of record.
At least 2 years of experience enforcing data-sharing policies that restrict user access to the entitled version of a data product at subscription.
Bachelor’s degree; the posting also specifies a BS in Computer Science, Data Science, or Information Systems.
At least 4 years of experience designing and implementing data, analytics, and AI solutions for large enterprises using data-platform tools from a cloud service provider.
At least 4 years of experience modeling and using relational data services, with proficiency in Python, PySpark, and SQL, including tuning SQL queries to extract data from sources such as relational databases, flat files, CSV, HTML, and JSON into parquet-format Lakehouse Medallion Architecture.
At least 4 years of programming or scripting experience with languages including Python, R, and Spark.
Experience designing and delivering a star-schema-based operational data store to enable visualization of program activities and outcomes, such as in Power BI.
Programming and scripting skills in SQL, Python, R, Spark, Terraform, DAGs, YAML, and Docker.

Nice To Haves

Proficiency with DevOps tools such as GitHub, Jenkins, Terraform, YAML, DAGs, Docker, Databricks, and SageMaker for building data and machine-learning pipelines that extract and store data from multiple sources in parquet destinations.
Experience promoting data engineering standards and best practices to encourage scalable, supportable data and machine-learning product delivery and use.
Experience implementing cloud-provider data governance and security tools to automate enforcement of data-sharing policies when subscribers access data products.

Responsibilities

Oversee the design, development, and maintenance of data pipelines that move, transform, and prepare data for analytics and operational systems.
Architect and optimize data infrastructure, including storage solutions, orchestration frameworks, and ETL/ELT processes.
Establish standards, monitoring systems, validation, and governance practices to ensure data quality, reliability, and security.
Lead and mentor data engineering staff by providing technical guidance, reviewing designs and code, and supporting professional growth.
Collaborate with data scientists, analysts, product teams, and business stakeholders to ensure data solutions meet organizational needs and strategic goals.
Serve as the primary State of Michigan technology lead working with the University of Michigan proof-of-concept team to support best-practice adoption in the state hosting environment.
Provide training to DAV data engineers.
Build and deliver data products from cloud data lakes, with traceability from product attributes and features to critical data elements in systems of record.
Enforce data-sharing policies so users can access only the data-product version to which they are entitled at the time of subscription.