Senior Data Engineer

University of Texas at Austin•Prior Lake, MN

65d•Remote

About The Position

The Senior Data Engineer for the UT Data Hub improves university outcomes and advances the UT mission to transform lives for the benefit of society by increasing the useability and value of institutional data. You will create complex data pipelines into UT’s cloud data ecosystem in support of academic and administrative needs. In collaboration with our team of data professionals, you will help build and run a modern data hub to enable advanced data-driven decision making for UT. You will leverage your creativity to solve complex technical problems and build effective relationships through open communication.

Requirements

Bachelor’s degree in Computer Science, Information Systems, Engineering, or equivalent professional experience.
At least two years of hands-on experience in Data Engineering using cloud-based platforms (AWS, Azure, or GCP) with emphasis on Databricks or Spark-based pipelines.
Proven experience in designing, building, and automating scalable, production-grade data pipelines and integrations across multiple systems and APIs.
Proficiency in Python and SQL, with demonstrated ability to write efficient, reusable, and maintainable code for data transformations and automation.
Strong knowledge of ETL/ELT principles, data lakehouse architectures, and data quality monitoring.
Experience implementing and maintaining CI/CD pipelines for data workflows using modern DevOps tools (e.g., GitHub Actions, Azure DevOps, Jenkins).
Familiarity with data governance, security, and compliance practices within cloud environments.
Strong analytical, troubleshooting, and performance optimization skills for large-scale distributed data systems.
Excellent communication and collaboration skills to work effectively with technical and non-technical stakeholders.
Demonstrated experience mentoring and guiding junior engineers or peers on technical projects.
Equivalent combination of relevant education and experience may be substituted as appropriate.

Nice To Haves

Five or more years of experience in Data Engineering or related fields with increasing technical leadership responsibilities.
Three or more years of experience developing and optimizing data pipelines on Databricks, including Delta Lake, Delta Live Tables, and Databricks Workflows.
Experience designing AI-ready data architectures and integrating data workflows with machine learning and analytics environments.
Experience with distributed data processing frameworks such as Spark, Kafka, or Flink.
Databricks or AWS certifications (e.g., Databricks Certified Data Engineer Professional, AWS Solutions Architect, or AWS Data Analytics Specialty).
Two or more years of experience in Agile software development environments, including use of tools such as JIRA, Confluence, or similar for issue tracking and project management.
Hands-on experience with data orchestration tools (e.g., Airflow, Databricks Workflows, or AWS Step Functions).
Exposure to data governance frameworks and AI/ML operations (MLOps) concepts such as MLflow or model monitoring.
Demonstrated ability to lead or supervise small teams or project-based technical efforts.
Passion for continuous learning and staying current with advancements in Databricks, cloud-based data engineering, and AI enablement.

Responsibilities

Data Engineering: Lead the design, development, and automation of scalable, high-performance data pipelines across institutional systems, AWS, Databricks, and external vendor APIs.
Implement Databricks Lakehouse architectures to unify structured and unstructured data, enabling AI-ready data platforms that support advanced analytics and machine learning use cases.
Build robust and reusable ETL/ELT workflows using Databricks, Spark, Delta Lake, and Python to support batch and streaming integrations.
Ensure performance, reliability, and data quality of data pipelines through proactive monitoring, optimization, and automated alerting.
Partner with business and technical stakeholders to define and manage data pipeline parameters—including load frequency, transformation logic, and delivery mechanisms—ensuring alignment with analytical and AI goals.
Ensure all data engineering solutions adhere to university security, compliance, and governance guidelines, while leveraging best practices in cloud-native data development.
Develop and maintain comprehensive technical documentation of data pipeline designs, data flows, and operational procedures.
Collaborate with enterprise data architects, data modelers, data stewards, and subject matter experts to ensure data consistency, lineage, and semantic alignment across the ecosystem.
Continuously evaluate and introduce emerging technologies—such as Databricks Unity Catalog, MLflow, Delta Live Tables, and AI-driven data observability tools—to enhance the data engineering landscape.
Drive innovation by modernizing existing pipelines toward AI-readiness, enabling future integration with predictive analytics and machine learning models.
Stay current with advances in Databricks, AI-driven data engineering, and cloud technologies, and advocate for their responsible adoption.
Contribute to the vision of building a modern, AI-ready data ecosystem that powers advanced analytics, automation, and decision intelligence across the University.
Collaboration, Support, & Communication: Work both independently and collaboratively within cross-functional teams to deliver data products and pipelines that meet the University’s evolving data and analytics needs.
Communicate clearly and effectively with technical and non-technical stakeholders regarding project progress, risks, dependencies, and technical challenges.
Promote collaboration and knowledge sharing within the Data Engineering team through brainstorming sessions, design reviews, and Databricks best-practice discussions.
Foster a culture of learning and innovation, supporting team morale and professional growth.
Provide mentorship and peer guidance to junior data engineers on data pipeline design, Databricks workflows, and coding best practices.
Participate in change management processes to ensure transparency and coordination across teams during system enhancements or platform migrations.
Perform other related functions as assigned.

Benefits

Competitive health benefits (Employee premiums covered at 100%; family premiums at 50%)
Vision, dental, life, and disability insurance options
Paid vacation, sick leave, and holidays
Teachers Retirement System of Texas (a defined benefit retirement plan)
Additional voluntary retirement programs: tax sheltered annuity 403(b) and a deferred compensation program 457(b)
Flexible spending account options for medical and childcare expenses
Training and conference opportunities
Tuition assistance
Athletic ticket discounts
Access to UT Austin's libraries and museums
Free rides on all UT Shuttle and Capital Metro buses with staff ID card