LillyDirect Pharmacy Data Engineer

Eli Lilly and Company • Us, IN

3d

About The Position

Lilly is seeking a Data Engineer for the LillyDirect Pharmacy Covered Entity (LDPCE) Master Data team. The role designs, builds, and maintains scalable data pipelines and infrastructure that support scientific and business initiatives. It ensures the availability, quality, and integrity of data used for analysis and business transactions, turning raw data into reliable, governed datasets for downstream teams.

The Data Engineer develops data storage, processing, and retrieval solutions, data models, and ETL/ELT workflows, and integrates data across multiple systems. The role supports LDPCE integrations with Reltio and Transcend and applies governance, privacy, security, and data quality practices to sensitive personal, health, and consent data. Working with technical and business partners, the engineer owns pipelines and work streams from design through production and contributes to architecture and design decisions.

The position also promotes coding and testing best practices, releases code through CI/CD, monitors and troubleshoots pipelines, supports production incidents and root-cause analysis, and improves performance, reliability, and cost. Lilly describes the work as purpose-driven and focused on helping deliver life-changing medicines.

Requirements

• Bachelor’s degree in Computer Science, Information Technology, Management Information Systems, or equivalent work experience.
• At least 5 years of data engineering experience using SQL, Python, PySpark, and cloud platforms such as AWS, Azure, or GCP, including services such as Lambda, Glue, EMR, S3, Redshift, Athena, and IAM roles/policies.
• At least 2 years of experience working in Agile environments, with hands-on GitHub and CI/CD pipeline experience for code deployment.
• Hands-on experience with Databricks, including workspace configuration, notebook development, Delta Lake, and performance optimization.
• At least 1 year of experience with orchestration tools such as Airflow or Databricks Workflows.
• Ability to collaborate with business stakeholders to translate requirements into scalable technical solutions.
• Strong communication and collaboration skills, problem-solving mindset, and attention to detail.

Nice To Haves

• Experience building high-performance, scalable pipelines using Data Lakehouse, Data Warehouse, and Data Mart standards, with a deep understanding of ETL/ELT workflows.
• Strong expertise in data modeling, managing large datasets, and implementing secure, compliant data governance practices.
• Working knowledge of data governance, security, and data quality practices, including handling sensitive PII/PHI in regulated environments.
• Pharmaceutical or healthcare industry experience, including familiarity with privacy requirements such as HIPAA.
• Hands-on Reltio or comparable MDM platform experience, including match and merge, survivorship, and integration patterns.
• Familiarity with consent and preference management, including opt-in/opt-out, purpose, and channel; experience with Transcend or a similar consent platform.
• Familiarity with streaming frameworks such as Kafka, Kinesis, or MSK.
• Experience with Unity Catalog, infrastructure as code such as Terraform, and data quality frameworks such as Great Expectations or dbt tests.
• AWS Certified Data Engineer – Associate certification.
• Databricks Certified Data Engineer (Associate or Professional) certification.

Responsibilities

• Design, build, and maintain scalable, reliable batch and near-real-time data pipelines using Python and PySpark on Databricks and AWS.
• Develop and optimize data models, ETL/ELT workflows, and integrations across systems and platforms using Delta Lake and lakehouse (medallion) patterns.
• Support integrations with Reltio and Transcend, including data loads, extracts, and reconciliation.
• Build data quality, lineage, and reconciliation checks to ensure profile and consent data is accurate, complete, and auditable.
• Implement data governance, security, and quality standards, including access controls, masking, and retention for sensitive PII/PHI and consent data.
• Collaborate with the Technical Lead, product owners, architects, analysts, and business stakeholders to understand requirements and deliver solutions.
• Own pipelines and work streams from design through production and contribute to architectural and design decisions.
• Promote coding, code review, automated testing, and deployment best practices, and release code through CI/CD pipelines.
• Monitor, troubleshoot, and improve pipeline performance, reliability, and cost; support production incidents and root-cause analysis.
• Automate manual processes and identify opportunities to optimize data workflows and reduce costs.
• Build privacy and consent protections into pipeline designs and raise risks early.
• Share expertise, mentor junior engineers, and communicate clearly with technical and non-technical stakeholders.
• Partner with and influence vendor resources on solution development to align on data and technical direction and delivery.

Benefits

• Company bonus, depending in part on company and individual performance.
• Eligibility to participate in a company-sponsored 401(k).
• Pension.
• Vacation benefits.
• Eligibility for medical, dental, vision, and prescription drug benefits.
• Flexible benefits, such as healthcare and/or dependent day care flexible spending accounts.
• Life insurance and death benefits.
• Certain time-off and leave-of-absence benefits.
• Well-being benefits, including an employee assistance program, fitness benefits, and employee clubs and activities.
• Employee resource groups open to all employees.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level