Senior Data Engineer

Elevance Health•Tampa, FL

8d•$127,200 - $228,960•Hybrid

About The Position

The Senior Data Engineer serves as the technical lead and platform owner for the security data lake supporting Application Security, Vulnerability Management, and related cybersecurity functions. This role defines the platform architecture, engineering standards, data models, integration patterns, governance controls, and technical roadmap necessary to deliver reliable and actionable security data. The role is also responsible for the design and delivery of secure, scalable, and governed data ingestion, transformation, enrichment, storage, and consumption capabilities across Microsoft Fabric and supporting Azure services. How you will make an impact Leads the architecture, engineering, and ongoing operation of the security data lake and associated data products supporting multiple security technologies and data sources. Establishes the technical roadmap, engineering standards, reusable design patterns, and platform operating model. Acts as a subject matter expert among peers, with manager and senior management on all matters pertaining to data platforms and the supporting architecture. Develop and maintain the end-to-end architecture of the security data lake, including ingestion, storage, transformation, enrichment, semantic, and consumption layers, based on business, security, compliance, scalability, and reliability requirements. Ensure data pipelines comply with security policies and regulatory requirements (e.g., HIPAA, NIST, SOC 2) Develop testing plans to ensure quality of implementation, including deployment validation, data quality checks, reconciliation, and repeatable testing of data pipeline changes. Determine and perform complex configuration changes to meet business and information security requirements, including Microsoft Fabric workspaces, data pipelines, connections, service principals, secrets, and environment-specific configuration. Provide trouble resolution on complex problems and lead implementations for data pipelines, Microsoft Fabric, Azure DevOps, REST API, and source-system integration issues. Provide technical support to business and technology associates, in risk assessments and implementation of appropriate information security procedures, standards and technologies. Represent major upgrades and reconfigurations in change control; design and analyze vendor and platform services meeting business and information security requirements, including governed promotion across development, test, and production environments. Represent infrastructure security support in significant projects and perform the most complex operations and administration tasks. Respond to level 3 & 4 change and problem requests without supervision; Lead level 1 & 2 incident recoveries and root cause analysis. Coordinate and prepare the reporting of data security events and incidents. Perform capacity analysis; recommend and implement capacity increases for data platform performance, scalability, and reliability. Maintains relationship with key vendors; Serve as the technical escalation for results of preventative maintenance routines; Supervise preventative maintenance; Creates presentations and seeks IT management approval and acceptance of significant replacements or reconfigurations of the data pipeline.

Requirements

BS/BA in information Technology or related field of study and a minimum of 8 years experience in systems administration and security aspects of information systems, access management and network security technologies, network communications, computer networking, telecommunications, systems development and management, hardware, software, data, and people; experience with multiple technical and business disciplines required; or any combination of education and experience, which would provide an equivalent background.

Nice To Haves

Experience with architecting and operating Microsoft Fabric, Apache Spark, or comparable enterprise lakehouse platforms
Experience in the design, development, testing, deployment and maintenance of scalable data pipelines for structured and unstructured data.
Experience with SQL, Python, PySpark, Spark, Delta Lake, Parquet, REST APIs, JSON, Git, Azure DevOps, automated testing, and CI/CD is strongly preferred.
Relevant Microsoft Fabric, Azure Data, Databricks, cloud architecture, or data-engineering certifications are preferred.
Security certifications such as CISSP, ISSAP, ISSEP, or equivalent certifications are also preferred.
Experience planning and designing highly complex systems is strongly preferred.

Responsibilities

Leads the architecture, engineering, and ongoing operation of the security data lake and associated data products supporting multiple security technologies and data sources.
Establishes the technical roadmap, engineering standards, reusable design patterns, and platform operating model.
Acts as a subject matter expert among peers, with manager and senior management on all matters pertaining to data platforms and the supporting architecture.
Develop and maintain the end-to-end architecture of the security data lake, including ingestion, storage, transformation, enrichment, semantic, and consumption layers, based on business, security, compliance, scalability, and reliability requirements.
Ensure data pipelines comply with security policies and regulatory requirements (e.g., HIPAA, NIST, SOC 2)
Develop testing plans to ensure quality of implementation, including deployment validation, data quality checks, reconciliation, and repeatable testing of data pipeline changes.
Determine and perform complex configuration changes to meet business and information security requirements, including Microsoft Fabric workspaces, data pipelines, connections, service principals, secrets, and environment-specific configuration.
Provide trouble resolution on complex problems and lead implementations for data pipelines, Microsoft Fabric, Azure DevOps, REST API, and source-system integration issues.
Provide technical support to business and technology associates, in risk assessments and implementation of appropriate information security procedures, standards and technologies.
Represent major upgrades and reconfigurations in change control; design and analyze vendor and platform services meeting business and information security requirements, including governed promotion across development, test, and production environments.
Represent infrastructure security support in significant projects and perform the most complex operations and administration tasks.
Respond to level 3 & 4 change and problem requests without supervision;
Lead level 1 & 2 incident recoveries and root cause analysis.
Coordinate and prepare the reporting of data security events and incidents.
Perform capacity analysis; recommend and implement capacity increases for data platform performance, scalability, and reliability.
Maintains relationship with key vendors;
Serve as the technical escalation for results of preventative maintenance routines;
Supervise preventative maintenance;
Creates presentations and seeks IT management approval and acceptance of significant replacements or reconfigurations of the data pipeline.