Staff Data Engineer (Azure Databricks Platform)
About The Position
CVS Health is seeking a hands-on Staff Data Engineer to lead the development, maintenance, and strategic evolution of its Azure Databricks platform in support of healthcare delivery initiatives. The role is positioned across platform engineering, cloud engineering, data governance, and developer enablement. The engineer will be accountable for platform architecture, implementation, reliability, performance, security, and operational excellence, while enabling data engineers with tools and practices that help them deliver business value. The role includes overseeing and directing two platform engineers. It requires collaboration with data engineering, business, security, networking, and infrastructure teams to maintain a scalable, reliable, and secure platform. The position is full-time and offers a base pay range of $130,295 to $260,590, plus eligibility for a CVS Health bonus, commission or short-term incentive program and an equity award target.
Requirements
- At least 8 years of experience in a hands-on engineering role focused on cloud data platforms, distributed systems, and data engineering.
- Deep hands-on experience administering Azure Databricks, including cluster management, job orchestration, Delta Lake, and Unity Catalog.
- Expert-level knowledge of Azure, including Azure Data Lake Storage (ADLS), Azure Active Directory (AAD), and networking components.
- Strong proficiency in Terraform, Python, Apache Spark/PySpark, SQL, and Shell scripting.
- Proven experience with Infrastructure as Code tools such as Terraform and building automated CI/CD pipelines for platform and data application deployments.
- Strong understanding of distributed computing principles and large-scale data system architecture.
- Exceptional analytical and problem-solving skills, including experience diagnosing and resolving complex platform-level issues.
- Excellent communication, documentation, and presentation skills, with the ability to influence technical and business stakeholders.
- Bachelor's or master's degree in Computer Science, Engineering, or a related field.
Nice To Haves
- Experience with other data technologies such as Azure Synapse, Kafka, or Snowflake.
- Familiarity with healthcare data and healthcare insurance feeds.
- Relevant certifications, such as Microsoft Certified: Azure Solutions Architect Expert, Databricks Certified Data Engineer Professional, or Databricks Certified Platform Administrator.
Responsibilities
- Oversee platform operational health and maintain or enhance monitoring, logging, and alerting solutions.
- Lead incident response and perform root cause analysis for platform-wide issues.
- Maintain and enhance core components of the Azure Databricks environment and define the platform roadmap, including adoption of features and practices such as Delta Lake, Unity Catalog, Databricks Workflows, and GitHub CI/CD.
- Manage enforcement and enhancement of data security and governance policies.
- Manage Unity Catalog to support fine-grained access control, data lineage, and compliance with internal and external regulations.
- Implement platform-level cost management and performance-tuning strategies, and provide guidance and tools to optimize Databricks clusters and workloads.
- Maintain and enhance automated provisioning, scaling, and management of Databricks workspaces and resources using Terraform and CI/CD pipelines.
- Mentor data engineers and Azure Databricks platform engineers; develop and advocate best practices, reusable patterns, and libraries; and provide technical support and training.
- Partner with data engineering and business teams to understand their needs and maintain and enhance a platform that supports them.
- Collaborate with security, networking, and other infrastructure teams to ensure a cohesive and secure environment.
- Oversee and direct the efforts of two platform engineers.
Benefits
- Medical, dental, and vision coverage.
- Paid time off.
- Retirement savings options.
- Wellness programs and other resources, based on eligibility.
- Eligibility for a CVS Health bonus, commission, or short-term incentive program.
- An award target in the company's equity award program.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
