Principal Software Engineer

Red River • Lowell, MA

1d • Hybrid

About The Position

Red Hat Core Business Platforms is looking for a strategic and visionary Principal Software Engineer to provide technical leadership for our Data Platform and Data Products team in MA. This role is central to defining the long-term technical roadmap and elevating Red Hat's data-driven culture and execution across the organization. You will be responsible for architecting, leading, and overseeing the development of next-generation data products that support Red Hat’s operational and analytical needs, with a critical focus on leading the charge on Agentic First Development for advanced Machine Learning and Artificial Intelligence initiatives. You will set the technical direction for delivering high-quality data products (code + data) that service the entire organization through an InnerSource collaboration model. Your deep expertise in distributed systems, modern software and data engineering practices, and system architecture will be instrumental in ensuring that our data assets are not only trustworthy but also optimized for building and deploying autonomous AI Agents on enterprise-grade platforms at massive scale.

Requirements

• 10+ years of progressive experience in Software Engineering, Data Engineering, or a related field, with a track record of architecting and delivering complex, large-scale data systems.
• Expert-level proficiency in SQL and a major programming language (e.g., Python, Java, Scala), and deep knowledge of distributed data processing frameworks.
• Proven track record in designing and deploying cloud-native data warehousing or data lake solutions at an enterprise scale (e.g., Snowflake, Databricks, BigQuery, S3/MinIO).
• Deep, authoritative understanding of advanced data modeling principles (e.g., Data Mesh, dimensional modeling, data vault, domain-driven design).
• Extensive experience defining and optimizing CI/CD pipelines, GitOps practices, and version control strategies for large engineering teams (e.g., GitLab, GitHub Actions, Jenkins).
• Expertise in containerization technologies like Docker and Kubernetes, and proven experience running high-performance data services and AI workloads on Microservices, Containers, and Platform (MCP) servers.
• Exceptional technical leadership, mentoring, and cross-functional collaboration skills, with a history of successfully championing InnerSource principles and driving the adoption of platform-level data products across multiple engineering organizations.

Nice To Haves

• Experience leading the architecture and deployment of complex, intelligent services or autonomous AI Agents at massive scale, demonstrating an understanding of Agentic First Development principles (e.g., using frameworks like LangChain, custom service agents).
• Deep expertise in streaming data architecture and technologies (e.g., Kafka, Spark Streaming).
• Significant experience defining and implementing data governance and lineage solutions (e.g., data catalogs, lineage tracking).
• Master’s degree or PhD in Computer Science, Engineering, or a related quantitative field.

Responsibilities

• Define and Champion the Architectural Roadmap: Architect the strategic evolution of existing source data pipelines to an ELT model of data ingestion, ensuring high efficiency, real-time capabilities, and cross-organizational adoption.
• Establish Data Architecture Standards: Lead the definition of architectural patterns for cleanly separating source-aligned data products from aggregate data products, enforcing domain separation, robust governance, and security across the entire data mesh.
• Drive Agentic First Data Product Strategy: Set the technical vision and standards for architecting, developing, and maintaining data products specifically optimized for consumption by autonomous AI Agents and Machine Learning models. This includes designing scalable feature store infrastructure, defining standardized feature sets, and ensuring enterprise-wide data lineage and versioning frameworks are robust, acting as the data foundation for agentic systems.
• Lead Technical Governance and Metadata Strategy: Establish best practices for richly decorating data products with metadata to support seamless knowledge transfer, mass adoption, and the responsible application of Machine Learning and AI Agents, including metadata specifically for defining agent capabilities and tool use.
• Oversee Compliance and Responsible Data Use: Define the strategy for tagging and classifying data assets to ensure they are used responsibly throughout the organization, architecting and implementing organization-wide solutions for masking or restricting access to meet global compliance standards.
• Cultivate Engineering Excellence: Mentor senior engineers, champion software engineering best practices, and drive improvements to the code release process to support CI/CD and a high-velocity InnerSource collaboration model.
• Drive Discoverability and Integration: Architect the data product catalog and integration strategy, ensuring data products are registered, easily discoverable, and seamlessly join with all other business data products using unified identifiers and keys.
• Establish Data Integrity Frameworks: Design and lead the implementation of automated, resilient, and proactive data quality testing and monitoring frameworks to guarantee data integrity for all business-critical applications and AI model training at scale.
• Lead AI Agent Deployment and Scaling: Serve as the strategic leader and subject matter expert for Agentic First Development, defining the methodology for building, deploying, and monitoring high-reliability, autonomous AI Agents and microservices, focusing on planning, tool integration, fault tolerance, and ultra-low latency.
• Optimize Cloud-Native Infrastructure: Define the strategy and work with DevOps teams to architect and optimize the deployment and management of data product services and AI workloads efficiently on Microservices, Containers, and Platform (MCP) servers, leveraging expert-level knowledge of Kubernetes and cloud-native principles for extreme scale and performance.

Benefits

• Comprehensive medical, dental, and vision coverage
• Flexible Spending Account - healthcare and dependent care
• Health Savings Account - high deductible medical plan
• Retirement 401(k) with employer match
• Paid time off and holidays
• Paid parental leave plans for all new parents
• Leave benefits including disability, paid family medical leave, and paid military leave
• Additional benefits including employee stock purchase plan, family planning reimbursement, tuition reimbursement, transportation expense account, employee assistance program, and more!

What This Job Offers

• Job Type: Full-time
• Career Level: Principal