Senior Data Engineering Developer

Geisinger•Work from home (Pennsylvania), PA

6d•Remote

About The Position

The Geisinger Research Data Core transforms and manages clinical and genomic data for research use. As part of a collaborative, multidisciplinary team, the Senior Data Engineering Developer develops and maintains data systems and pipelines that support research applications and large-scale clinical and genomic datasets. The role contributes expertise in data architecture, data modeling, and application development and participates in the development life cycle as needed for project success. The position focuses on cloud-based data engineering, particularly with Databricks and AWS. It models clinical data in Databricks, develops and maintains scalable pipelines, and modernizes legacy ETL processes for cloud environments. It also supports data quality, performance optimization, monitoring, and general Databricks computing needs for research and analytical applications. The role includes technical design and architecture work, implementation and documentation, code review and testing, deployment and post-launch support, communication with the team and management, and training and mentoring staff. Work is typically performed in an office environment. The position is based work from home in Pennsylvania, with a day shift and 40 scheduled weekly hours. The role is regular and exempt. Relevant experience may combine related work experience and education: an associate’s degree equals two years, a bachelor’s degree equals four years, and a master’s degree equals six years.

Requirements

At least 10 years of related work experience.
Proven experience with Databricks and cloud-based data engineering platforms.
Strong experience with SQL and Python/PySpark for developing and optimizing data processing pipelines.
Experience with Apache Spark or other distributed data processing technologies.
Experience designing and implementing data models and data warehouse schemas for large-scale datasets.
Experience developing, maintaining, or modernizing ETL processes and data pipelines.
Experience working with relational databases and SQL Server.
Familiarity with AWS cloud services and cloud-based data architectures.
Experience with data quality, validation, monitoring, logging, and troubleshooting practices.
Understanding of data pipeline performance optimization and scalable data processing.
Excellent communication skills for conveying progress, roadblocks, technical issues, and solutions to team members and management.
Ability to work effectively on a multidisciplinary team and collaborate with technical and research stakeholders.
High school diploma or equivalent (GED) required.

Nice To Haves

Bachelor’s degree in a technology-related field preferred.
Experience migrating or modernizing on-premises ETL processes preferred.

Responsibilities

Design, develop, and maintain PySpark and SQL data pipelines that transform clinical data for research use.
Develop and maintain data models for clinical and other research data in the Databricks environment.
Review, analyze, modify, create, debug, design, and test data ingestion and processing pipelines using modern development methodologies and tools.
Define and optimize data warehouse schemas and data models for efficient querying, storage, and analytical workflows.
Ensure the scalability, reliability, and maintainability of data pipelines and architecture supporting large clinical and research datasets.
Implement and monitor data quality checks and validation processes to ensure data accuracy, completeness, and integrity throughout pipelines.
Troubleshoot and optimize Spark and SQL workloads, data pipelines, and Databricks compute resources.
Modernize on-premises Microsoft SQL Server ETL processes by designing and implementing equivalent cloud-based pipelines and workflows in Databricks.
Provide Databricks computing support, including helping users select compute resources and troubleshoot and optimize workloads.
Participate in design and application architecture discussions and help develop and document technical approaches and solutions for stakeholders.
Manage the process and life cycle of creating software that helps users complete computer tasks or programs.
Implement code and document system changes according to assignments and timelines.
Develop and follow department coding and user interface guidelines.
Provide code review, testing, debugging, technical documentation, testing instructions, go-live planning and assistance, and post-go-live support.
Communicate progress, roadblocks, and issues to the team and management in a timely manner.
Collaborate with team members to identify and resolve issues, improve system performance, and ensure code quality.
Provide training and documentation to team members on new development assignments.
Train and mentor staff members.
Assist in developing and documenting technical approaches for solutions for team stakeholders.