Senior Data Engineer

PrePass•Phoenix, AZ

About The Position

PrePass is seeking a Senior Data Engineer to architect, build, and scale its core data platform. The role leads the design of high-throughput data pipelines and lakehouse architectures while mentoring fellow engineers in modern software craftsmanship. The successful candidate will apply software-engineering rigor to data engineering, using strict Test-Driven Development (TDD), extreme programming (XP) practices, and AI-augmented workflows to deliver reliable, observable, mission-critical data systems in Databricks. The position focuses on building scalable Azure ETL/ELT pipelines, modernizing legacy SSIS integrations, and improving production reliability and data quality. The Senior Data Engineer will also help define team-wide engineering standards, architecture, documentation, and continuous improvement practices in an Agile environment.

Requirements

Proven experience designing, building, and operating distributed data pipelines and cloud data platforms at scale.
Hands-on expertise in TDD for data pipelines, including writing unit, integration, and data-contract tests before implementation.
Experience with XP practices such as pairing, refactoring, and continuous integration.
Practical experience integrating AI coding assistants and LLM-based tools into daily engineering workflows to accelerate delivery, improve test coverage, and automate documentation.
Strong experience operating high-availability production systems with clear uptime, reliability, and observability expectations.
Experience driving engineering standards, architectural reviews, documentation, and mentorship within an Agile team.

Nice To Haves

Experience in data science or machine learning, particularly MLOps, model deployment, Feature Store management, or feature engineering on Databricks.
Experience with Infrastructure as Code, such as Terraform or Bicep, for managing Databricks workspaces and cloud resources.
Familiarity with event-driven architecture, including queues, batches, and pub/sub models.
Production experience with Databricks, including Delta Lake, PySpark/Spark SQL, Databricks Workflows, and Unity Catalog.
Background working with high-volume IoT, telematics, or transportation industry data.

Responsibilities

Design, build, and operate scalable ETL/ELT pipelines on Azure to ingest, transform, and integrate data from multiple sources.
Use advanced SQL, Apache Spark, Databricks, and Delta Lake to support analytics, reporting, and real-time operational workloads.
Partner with the team to define and execute a strategy for re-architecting legacy SSIS workloads into modern cloud-native pipelines.
Lead proof-of-concept and prototyping work and turn findings into clear recommendations.
Model clean, modular, maintainable coding practices using strict TDD, pair programming, continuous refactoring, and automated CI/CD pipelines.
Use AI-assisted development tools to accelerate code generation, automate unit and data-quality test creation, streamline legacy refactoring, and enhance pipeline anomaly detection.
Operate and monitor mission-critical data systems against SLAs for data freshness, uptime, and reliability.
Diagnose and resolve integration and data-quality failures; build end-to-end traceability and observability; and tune pipeline and compute performance for cost and scalability.
Define and evolve team-wide data engineering standards, architectural documentation, and continuous improvement initiatives within an Agile environment.
Mentor fellow engineers.

Benefits

Medical, dental, and vision benefits beginning on the date of hire.
Paid time off, including vacation, sick leave, holidays, and floating holidays.
401(k) plan with employer match.
Company-funded lifestyle account for physical and mental well-being.
Tuition reimbursement program.
Voluntary benefits, including legal and pet discounts.
Employee Assistance Program at no cost.
Company-sponsored and funded Culture Team focused on employees’ physical, mental, and professional well-being.
Community give-back initiatives.
Employee development initiatives.