Principal Data Engineer

Caterpillar Inc.•Peoria, IL

1d•$128,470 - $208,770•Onsite

About The Position

This position will be responsible for building scalable, high performance data pipelines and data driven predictive analytics applications using automation, artificial intelligence (AI) and machine learning (ML) models to provide actionable recommendations and insights. The position will be part of Caterpillar’s Legal, Security and Public Policy team, tackling challenges and problems that are critical to realizing business value and mitigating high impact enterprise risks. This position will work with data scientists, business analysts, and others as part of a team that assembles large, complex data sets, automates processes and pipelines, builds models and provides insight to key stakeholders.

Requirements

BS in Computer Science, Computer Engineering, Data Engineering, Data Science, or related field.
5+ years development experience with modern development languages and frameworks, preferably Python, SQL, etc.
Understanding of data structures, algorithms, profiling & optimization.
Understanding and appreciation for the data governance standards, the dimensions of data quality, and data security. Familiarity with tools and frameworks to support these standards.
Experience managing a Snowflake database/environment. Understanding of SQL, ETL/ELT design, data modeling techniques, change data capture techniques. Familiarity with indexing, partitioning, and query optimization.
Experience building ETL/ELT pipelines using industry platforms (Snowflake, EMR, Glue, Snap Logic or similar). Experience building medallion, data lake, or lake house architectures.
Experience with version control in Github and devops practices, like continuous integration and continuous delivery (CI/CD)
Good communication, interpersonal, and collaboration skills. Ability to communicate difficult or technical concepts in a relatable manner. Ability to collaborate and prioritize cross functionally to achieve results.
Ability to guide and mentor other in coding and design best practice.

Nice To Haves

MS in Computer Science, Computer/Data Engineering, Data Science, or related fields.
Experience in agent assisted programming.
Experience building and deploying machine learning (ML), artificial intelligence (AI) and/or agentic AI solutions in a business context.
Deep Snowflake expertise including Cortex, Snowpipe, Snowpark, Snowsight, etc.
Deep experience in AWS EMR, Glue, Lambda, or similar.
Experience with modern agentic AI platforms (Cortex, Bedrock/Agent core, Dataiku, etc.)
Experience with trade and/or compliance data.
Experience in PowerBI, Power Apps, streamlit, or similar tools.

Responsibilities

Design, build, and maintain scalable Snowflake-based data platform supporting analytics, reporting, and AI/ML workloads
Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing data pipelines for greater scalability
Perform debugging, troubleshooting, modification and testing of integration solutions
Operationalize developed jobs, processes and models
Create solutions and methods to monitor systems and solutions
Review and optimize code. Automate code testing and pipelines
Design, develop, and maintain efficient and scalable AI and ML solutions
Put in place systems and processes to support data governance capabilities
Engage with business partners to participate in design and development of data integration/transformation solutions per functional requirements
Engage and actively seek industry perspectives through, continuous learning, peer groups, etc.