Data Engineer III

Valenz•Phoenix, AZ

1d•Remote

About The Position

As a Data Engineer III, you’ll be responsible for designing, building, and evolving scalable data systems that power analytics, product, and operational decision-making across the organization. You will operate as a senior individual contributor with end-to-end ownership of complex data initiatives, contributing directly to the architecture and evolution of our Databricks-based Lakehouse platform on Azure.

Requirements

4+ years of experience in data engineering or a related field, with a track record of delivering production-grade data systems
Strong hands-on experience with Databricks, Spark/PySpark, and distributed data processing at scale
Deep understanding of Delta Lake and modern Lakehouse architecture patterns
Proficiency in Python and SQL for large-scale data transformation and performance optimization
Proven experience building incremental, idempotent, and highly reliable data pipelines
Strong experience diagnosing and optimizing Spark workloads (partitioning strategies, AQE, caching, file sizing, query tuning)
Experience designing data models for analytics and downstream consumption (medallion architecture, dimensional modeling, or similar)
Experience implementing data quality, validation, and observability frameworks in production environments
Familiarity with CI/CD, version control, and modern DataOps practices
Experience supporting or integrating with machine learning workflows (feature pipelines, model inputs/outputs, or ML lifecycle support)
Familiarity with AI/ML concepts as applied to data engineering (intelligent ingestion, anomaly detection, automation)
Demonstrated ability to evaluate and adopt new technologies within cloud ecosystems (Azure, Databricks)
Strong communication skills and ability to collaborate with both technical and non-technical stakeholders

Nice To Haves

Familiarity with event-driven architectures (e.g., streaming, message queues, or event hubs)
Experience working with healthcare data (claims, eligibility, provider, or clinical datasets

Responsibilities

Own the design and implementation of scalable, production-grade data pipelines using Databricks, PySpark, SQL, and Python Operationalize machine learning workflows and feature pipelines.
Own and deliver complex, cross-functional data initiatives end-to-end, from ingestion and data modeling through production deployment and ongoing monitoring.
Design robust, reusable ETL frameworks using Delta Lake best practices (incremental processing, merge/upserts, schema evolution).
Diagnose and resolve performance challenges in distributed Spark workloads (data skew, shuffle, memory pressure, inefficient execution plans).
Build and enforce strong data quality practices, including validation frameworks, observability, and automated alerting.
Design and evolve data models across medallion architecture layers to support analytics and downstream applications.
Implement modern data ingestion patterns, including API-driven, event-based, and AI-assisted ingestion workflows.
Partner with analytics, architecture, and engineering teams to support advanced data use cases, including feature engineering and emerging machine learning workflows.
Evaluate and adopt new capabilities within Azure and Databricks (e.g., MLflow, Unity Catalog enhancements, platform optimizations) to improve scalability and developer productivity.
Contribute to architectural decisions and platform standards, balancing short-term delivery with long-term maintainability.
Write high-quality, well-tested, and maintainable code; lead by example through thoughtful code reviews.
Act as a go-to resource for diagnosing and resolving complex production issues across systems.
Mentor and elevate other engineers through collaboration, design discussions, and technical guidance.
Perform other duties as assigned.
Reasonable accommodation may be made to enable individuals with disabilities to perform essential duties.

Benefits

Generously subsidized company-sponsored Medical, Dental, and Vision insurance, with access to services through our own products, Healthcare Blue Book and KISx Card.
Spending account options: HSA, FSA, and DCFSA
401K with company match and immediate vesting
Flexible working environment
Generous Paid Time Off to include vacation, sick leave, and paid holidays
Employee Assistance Program that includes professional counseling, referrals, and additional services
Paid maternity and paternity leave
Pet insurance
Employee discounts on phone plans, car rentals and computers
Community giveback opportunities, including paid time off for philanthropic endeavors