Principal Software Engineer – Physical AI, Autonomy & Data Platform Engineering

Caterpillar•Chicago, IL

5d•Onsite

About The Position

Caterpillar Inc. is seeking a highly experienced Principal Software Engineer to lead the technical strategy and engineering execution for large-scale data ingestion and processing platforms supporting physical AI and autonomous systems. This role is responsible for driving engineering excellence across distributed scrum teams while designing scalable, cloud-native solutions for ingesting and processing high-volume sensor and telematics data including LiDAR, radar, video, image, and vehicle telemetry streams. The Principal Software Engineer will partner closely with Principal Data Architects, Product Owners, and Engineering Leadership to define reusable data platform capabilities that support advanced analytics, machine learning, and autonomy initiatives. This is a hands-on technical leadership role with significant influence on platform architecture, engineering standards, scalability strategy, and long-term technology direction. The ideal candidate combines deep expertise in software engineering, distributed systems, cloud architecture, and SDLC discipline with the ability to lead engineering efforts in highly ambiguous and rapidly evolving technical domains. This role operates at the frontier of physical AI and autonomy engineering, where technologies, architectural patterns, and best practices are continuously evolving. Success in this position requires an engineer who thrives in ambiguity, adapts quickly to emerging technologies, and can drive progress despite incomplete or constantly changing information. The Principal Software Engineer will serve as a technical anchor and role model for software and data engineering teams, helping establish a culture that embraces experimentation, iterative development, and continuous learning. This individual must be highly effective operating in “the grey” — balancing strategic architectural thinking with pragmatic execution in a frontier engineering environment.

Requirements

Extensive knowledge of the decision-making process and associated tools and techniques; ability to accurately analyze situations and reach productive decisions based on informed judgment.
Superior understanding of effective communication concepts, tools and techniques; ability to effectively transmit, receive, and accurately interpret ideas, information, and needs through the application of appropriate communication behaviors.
Expert knowledge of software development tools and activities; ability to produce software products or systems in line with product requirements.
Expert knowledge of software development life cycle; ability to use a structured methodology for delivering and managing new or enhanced software products to the marketplace.
Extensive knowledge of software integration processes and functions; ability to design, develop and maintain interfaces and linkage to alternative platforms and software packages.
Extensive knowledge of software product design; ability to convert business requirements into the software product design.
Extensive knowledge of technical aspects of a software products; ability to design, configure and integrate technical aspects of software products.
Extensive knowledge of software product testing; ability to design, plan, and execute testing strategies and tactics to ensure software product quality and adherence to stated requirements.
Bachelor’s or Master’s degree in Computer Science, Software Engineering, Computer Engineering, or related field.
10+ years of software engineering experience with significant experience in principal, staff, or lead-level technical leadership roles.
Expert-level proficiency in Python and/or Java.
Deep expertise in system design, distributed systems, and large-scale cloud-native architectures.
Strong experience designing and implementing streaming and batch data processing systems at enterprise scale.
Hands-on experience with AWS cloud services and architecture patterns; experience with Azure or GCP also valued.
Proven experience building scalable ingestion pipelines for high-volume structured and unstructured data.
Experience working with sensor-based or telemetry data domains such as LiDAR, radar, video, imagery, IoT, or vehicle telematics.
Strong understanding of modern data lake/lakehouse architectures and medallion (Bronze/Silver/Gold) data modeling patterns.
Experience with technologies such as Kafka, Kinesis, Spark, Flink, Airflow, Databricks, EMR, or equivalent platforms.
Strong understanding of CI/CD pipelines, DevOps practices, automated testing, infrastructure as code, and operational excellence.
Experience leading technical strategy across multiple engineering teams in agile environments.
Demonstrated success operating effectively in highly ambiguous or rapidly evolving technical environments.
Strong comfort level with experimentation, prototyping, and iterative architecture refinement.
Ability to make sound technical decisions with incomplete information and evolving constraints.

Nice To Haves

Experience supporting AI/ML, autonomous systems, computer vision, robotics, or advanced analytics platforms.
Familiarity with geospatial data processing and high-throughput sensor fusion pipelines.
Experience with containerization and orchestration technologies such as Docker and Kubernetes.
Knowledge of data governance, lineage, security, and compliance best practices.
AWS, Azure, or GCP cloud certifications.
Experience implementing observability, site reliability engineering (SRE), and platform reliability practices.
Passion for advancing technologies in physical AI, autonomy, sensor intelligence, and large-scale data systems.

Responsibilities

Lead engineering efforts in emerging domains related to physical AI, autonomy, and next-generation sensor-driven systems.
Operate effectively in environments with evolving requirements, incomplete datasets, and rapidly changing technology landscapes.
Drive iterative development practices that enable rapid experimentation, feedback loops, and continuous platform evolution.
Guide engineering teams through technical uncertainty by decomposing ambiguous problems into actionable engineering strategies.
Foster a culture of innovation, adaptability, resilience, and continuous learning across engineering organizations.
Evaluate emerging technologies, frameworks, and architectural approaches to support long-term platform evolution.
Partner with architects, researchers, and product leaders to translate innovative concepts into scalable production systems.
Establish engineering patterns that support agility while maintaining scalability, reliability, and long-term maintainability.
Design and oversee implementation of scalable ingestion pipelines for LiDAR, radar, video, image, and telematics data.
Partner with Principal Data Architects to design reusable data products and domain-oriented data models.
Architect and optimize Bronze, Silver, and Gold data layer pipelines supporting both streaming and batch processing workloads.
Ensure data pipelines are performant, fault tolerant, observable, secure, and cost optimized.
Drive metadata, lineage, governance, and reusable data object standards across the platform.
Enable downstream analytics, AI/ML, computer vision, and operational use cases through robust data engineering practices.
Design reusable ingestion and transformation frameworks capable of supporting rapidly evolving autonomy workloads.
Design and implement highly scalable solutions on AWS or comparable cloud platforms such as Azure or GCP.
Lead adoption of cloud-native architectures including microservices, event-driven systems, and distributed processing frameworks.
Architect real-time and near-real-time streaming solutions using technologies such as Kafka, Kinesis, Spark Streaming, Flink, or equivalent.
Design large-scale batch processing frameworks for high-throughput data workloads.
Optimize infrastructure for scalability, resiliency, latency, observability, and cost efficiency.
Drive architectural decisions supporting large-scale distributed compute and storage systems.
Lead development efforts using Python and/or Java in enterprise-scale environments.
Champion SDLC discipline including CI/CD, automated testing, infrastructure as code, code quality, release management, and operational maturity.
Establish engineering practices supporting reliability, observability, maintainability, and platform stability.
Participate in hands-on development, prototyping, troubleshooting, and performance tuning of critical platform components.
Drive modernization initiatives and continuous improvement of engineering processes and platform capabilities.
Promote iterative engineering practices that balance rapid innovation with production-grade engineering discipline.
Work closely with Product Management, Data Engineering, ML Engineering, Platform Engineering, and DevOps teams.
Translate business and platform objectives into actionable technical roadmaps for scrum teams.
Provide technical leadership across multiple agile teams and ensure alignment to architectural strategy.
Facilitate technical design reviews, sprint planning, backlog refinement, dependency management, and engineering governance activities.
Influence engineering culture by promoting collaboration, accountability, adaptability, and engineering rigor.