Senior Data Engineer

8451 • Chicago, OH

58d

About The Position

As a Senior Data Engineer , you will have the opportunity to build solutions that ingest, transform, store, and distribute our big data to be consumed by data scientists and our products. Our data engineers use PySpark/Python, Databricks, and other Azure based data engineering technologies and visualization tools to deliver data capabilities and services to our scientists, products, and tools.

Requirements

• 3+ years proven ability of professional Data Development experience
• 2+ years proven ability of developing with Databricks or Hadoop/HDFS
• 2+ years of experience with PySpark/Spark
• 3+ years of experience developing with either Python or Java
• 3+ years of experience with SQL and NoSQL technologies.
• Full understanding of ETL concepts and Data Warehousing concepts
• Experience with Pipeline Orchestration
• Experience with CI/CD
• Experience with version control software
• Strong understanding of Agile Principles (Scrum)
• Bachelor’s Degree (Computer Science, Management Information Systems, Mathematics, Business Analytics, or STEM)

Nice To Haves

• Experience with Azure
• Experience with Databricks Delta Tables, Delta Lake, Delta Live Tables
• Proficient with Relational Data Modeling
• Experience with CI/CD pipelines
• Experience with Python Library Development
• Experience with Structured Streaming (Spark or otherwise)
• Experience with Kafka and/or Azure Event Hub
• Experience with Github SaaS / Github Actions
• Experience with Snowflake
• Exposure to BI Tooling (Tableau, Power BI, Cognos, etc.)

Responsibilities

• Take ownership of features and drive them to completion through all phases of the entire 84.51° SDLC. This includes internal and external facing applications as well as process improvement activities:
• Design, build, and maintain scalable, efficient, and reliable data pipelines to support data ingestion, transformation, and integration across diverse sources and destinations, using tools such as Kafka, Databricks, and similar toolsets
• Implement automated unit and integration testing
• Collaborate with architecture and lead engineers to ensure consistent development practices
• Provide mentoring to junior engineers
• Participate in retrospective reviews
• Participate in the estimation process for new work and releases
• Collaborate with other engineers to solve and bring new perspectives to complex problems
• Drive improvements in data engineering practices, procedures, and ways of working
• Embrace new technologies and an ever-changing environment

Benefits

• Health: Medical: with competitive plan designs and support for self-care, wellness and mental health.
• Dental: with in-network and out-of-network benefit.
• Vision: with in-network and out-of-network benefit.
• Wealth: 401(k) with Roth option and matching contribution.
• Health Savings Account with matching contribution (requires participation in qualifying medical plan).
• AD&D and supplemental insurance options to help ensure additional protection for you.
• Happiness: Paid time off with flexibility to meet your life needs, including 5 weeks of vacation time, 7 health and wellness days, 3 floating holidays, as well as 6 company-paid holidays per year.
• Paid leave for maternity, paternity and family care instances.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level