Senior Data Engineer

PrePass•Phoenix, AZ

About The Position

PrePass is seeking a Senior Data Engineer to architect, build, and scale its core data platform. The role leads the design of high-throughput data pipelines and lakehouse architectures and mentors fellow engineers in modern software craftsmanship. It emphasizes treating data engineering with the rigor of software engineering, using strict Test-Driven Development (TDD), extreme programming (XP) practices, and AI-augmented workflows to deliver reliable, observable, mission-critical data systems in Databricks. The engineer will build and operate scalable Azure ETL/ELT pipelines, modernize legacy SSIS integrations, champion TDD and XP practices, apply AI-assisted development tools, and own production reliability and data quality. The role also helps define engineering standards, architectural documentation, and continuous improvement initiatives in an Agile environment.

Requirements

Proven track record designing, building, and operating distributed data pipelines and cloud data platforms at scale.
Hands-on expertise in TDD for data pipelines, including writing unit, integration, and data-contract tests before implementation.
Experience with XP practices such as pairing, refactoring, and continuous integration.
Practical experience integrating AI coding assistants and LLM-based tooling into daily engineering workflows to accelerate delivery, improve test coverage, and automate documentation.
Strong experience operating high-availability production systems with clear uptime, reliability, and observability expectations.
Experience driving engineering standards, architectural reviews, documentation, and mentorship within an Agile team.

Nice To Haves

Experience in Data Science or Machine Learning, particularly MLOps, model deployment, Feature Store management, or feature engineering on Databricks.
Experience with Infrastructure as Code, such as Terraform or Bicep, for managing Databricks workspaces and cloud resources.
Familiarity with event-driven architecture, including queues, batches, and pub/sub models.
Production experience with Databricks, including Delta Lake, PySpark/Spark SQL, Databricks Workflows, and Unity Catalog.
Background in high-volume IoT, telematics, or transportation industry data.

Responsibilities

Design, build, and operate scalable ETL/ELT pipelines on Azure to ingest, transform, and integrate data from multiple sources.
Use advanced SQL, Apache Spark, Databricks, and Delta Lake to support analytics, reporting, and real-time operational workloads.
Partner with the team to define and execute a strategy for re-architecting legacy SSIS workloads into modern, cloud-native pipelines.
Lead proof-of-concept and prototyping work for legacy integration modernization and turn findings into clear recommendations.
Lead by example in writing clean, modular, and maintainable code using strict TDD, pair programming, continuous refactoring, and automated CI/CD pipelines.
Use AI-assisted development tools to accelerate code generation, automate unit and data-quality test creation, streamline legacy refactoring, and enhance pipeline anomaly detection.
Operate and monitor mission-critical data systems against SLAs for data freshness, uptime, and reliability.
Diagnose and resolve integration and data-quality failures; build end-to-end traceability and observability; and tune pipeline and compute performance for cost and scalability.
Define and evolve team-wide data engineering standards, architectural documentation, and continuous improvement initiatives within an Agile environment.
Mentor fellow engineers.

Benefits

Medical, dental, and vision benefits starting on the date of hire.
Paid time off, including vacation, sick leave, holidays, and floating holidays.
401(k) plan with employer match.
Company-funded lifestyle account upon hire for physical and mental well-being expenses.
Tuition reimbursement program.
Voluntary benefits, including legal and pet discounts.
Employee Assistance Program at no cost.
Company-sponsored and funded Culture Team focused on employees’ physical, mental, and professional well-being.
Community give-back initiatives.
Employee development initiatives.