Senior Data Engineer
About The Position
Join the Magnus Lake House team within Optum Insight as a Senior Data Engineer. In this role, you will be a key contributor to a high-profile migration project, transitioning our legacy Cornerstone Data Warehouse to a modern cloud-based Lakehouse architecture. This position offers a hybrid work environment based out of our Eden Prairie, MN or Raleigh, NC offices. You will be responsible for designing diverse ingestion patterns to bring data into the lakehouse and developing sophisticated solutions to share that data with consumers. We are looking for a hands-on developer who is eager to use modern AI-assisted tools to drive productivity and innovation in a fast-paced environment. You’ll enjoy the flexibility to telecommute from anywhere within the U.S. as you take on some tough challenges.
Requirements
- Bachelor’s degree in Computer Science, Mathematics, or a related field; OR a High School Diploma/GED with 4+ years of equivalent data engineering experience
- 5+ years of experience in data engineering or software development
- 3+ years of professional programming expertise in either Python or Java, with the demonstrated ability to read and modify code in both languages
- 2+ years of experience building and orchestrating data pipelines in an Azure environment (ADF, Databricks)
- 1+ year of experience utilizing AI-assisted development tools (e.g., GitHub Copilot, Codex) to enhance development productivity
Nice To Haves
- Experience with legacy data warehouse migrations and modernizing data architectures
- Proven experience designing and implementing data-sharing patterns for downstream consumers
- Deep understanding of Medallion Architecture and Delta Lake
- Experience with CI/CD practices and version control in a cloud-native environment (e.g., Azure DevOps, GitHub)
- Hands-on experience using the dbt (data build tool) framework for data transformation
- Ability to work effectively in a hybrid team environment, balancing independent development with collaborative design sessions
Responsibilities
- Use enterprise-approved AI tools and coding assistants to streamline development workflows, automate complex tasks, and drive continuous improvement in code quality and efficiency
- Design and implement robust data ingestion patterns to migrate legacy Cornerstone Data Warehouse assets into the Magnus Lake House
- Develop and deploy scalable data-sharing solutions to provide consumers with efficient access to data using various architectural patterns
- Build and manage automated data pipelines using Azure Data Factory (ADF) to ingest and process files within Databricks
- Utilize the dbt (data build tool) framework to execute complex data transformations following Medallion Architecture (Bronze, Silver, and Gold layers)
- Interpret, maintain, and enhance existing codebases in both Java and Python to support project requirements
- Evaluate emerging trends in data engineering and AI-assisted development to inform solution design and identify opportunities for strategic innovation
- Collaborate with cross-functional teams to ensure data pipelines meet security, scalability, and performance standards
Benefits
- Comprehensive benefits package
- Incentive and recognition programs
- Equity stock purchase
- 401k contribution
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
