Principal Data Engineer

Raymond James•Saint Petersburg, FL

1d•Hybrid

About The Position

We are seeking a highly experienced Principal Data Engineer to provide technical leadership for mission-critical Security/Product Master Data Platforms and other enterprise-scale data platforms that support the entire enterprise. This role requires deep data engineering expertise, strong database and platform engineering skills, hands-on experience with Oracle, Redshift, Python, Spark, Glue, AWS EMR, and Iceberg, and proficiency using AI to improve engineering productivity, solution quality, and delivery velocity. The candidate will be expected to lead modernization of enterprise data capabilities toward cloud data lakehouse and medallion architecture while maintaining operational stability, resiliency, performance, security, and enterprise availability. As a Principal Engineer, you will act as the technical authority for enterprise-wide platforms that manage and distribute critical security, product, and other high-value enterprise data used across business, operations, analytics, regulatory, and downstream application capabilities. The ideal candidate combines strong functional understanding of master data and enterprise data domains with hands-on engineering depth, AI-enabled engineering practices, and experience transitioning legacy or operational data platforms toward a modern cloud data lakehouse architecture using medallion patterns.

Requirements

15+ years of progressive engineering experience, including technical leadership for enterprise-scale data, database, or platform engineering capabilities.
Strong hands-on data engineering experience with Python, Oracle/ODI, Spark, AWS EMR, Redshift, Iceberg, and large-scale data processing patterns.
Experience developing, supporting, maintaining, and modernizing mission-critical database, master data, or enterprise-scale data platforms in production environments.
Strong functional and data understanding of Security/Product Master Data Platforms, with working knowledge of enterprise data domains such as Clients, Accounts, Assets and Liabilities, Trades, and Activities.
Experience with cloud data lakehouse architecture, medallion patterns, data quality controls, lineage, governed consumption, and reusable data product patterns.
Proficiency using AI tools and AI-assisted engineering practices to improve productivity, solution quality, documentation, troubleshooting, and delivery velocity.
Strong communication, technical influence, and cross-functional leadership skills with the ability to engage engineering, architecture, governance, operations, product, and business stakeholders.
Experience in Agile/Scrum at scale, such as SAFe or similar frameworks.
Experience in financial services, wealth management, capital markets, Security/Product Master Data, or enterprise reference data platforms.
Experience leading modernization of enterprise master data or reference data platforms from legacy technologies toward cloud data lakehouse, medallion architecture, or governed data product models.
Experience with enterprise data governance, data quality, lineage, stewardship, metadata management, and enterprise consumption patterns at scale.
Experience applying AI-enabled engineering practices to improve development efficiency, operational productivity, solution quality, and delivery velocity.

Nice To Haves

Expert-level proficiency in data engineering and large-scale data processing using Python, Oracle/ODI, Spark, AWS EMR, Redshift, Iceberg, distributed systems design, and modern platform engineering patterns.
Deep expertise in CI/CD using Jenkins, Kubernetes, and containerization.
Strong experience designing, building, and operating data pipelines, APIs, batch and near-real-time data processing capabilities, data quality controls, lineage, and enterprise consumption patterns.
Experience modernizing legacy or operational platforms into cloud data lakehouse architecture using medallion patterns, including bronze, silver, and gold layers, curated data products, and governed consumption.
Strong functional understanding of enterprise master data domains, especially Security/Product Master, with exposure to Clients, Accounts, Assets and Liabilities, Trades, and Activities.
Advanced knowledge of database performance optimization, data modeling, platform reliability, and production support for mission-critical data platforms.
Strong understanding of cloud platforms, including AWS, Azure, and GCP, and modern architecture patterns.
Experience in enterprise architecture, system integration, and platform engineering.

Responsibilities

Establish engineering standards, data architecture patterns, integration patterns
Define and drive the target-state architecture for enterprise Security/Product, and design principles for mission-critical master data and enterprise-scale data platforms.
Lead system design and modernization roadmaps for high-impact initiatives across security, product, master data, and other enterprise-scale data domains.
Provide technical leadership across multiple teams, not limited to a single project or squad.
Act as a trusted advisor to leadership on technology strategy, trade-offs, and long-term platform evolution.
Drive alignment across engineering, data, and platform teams to ensure consistency and reusability.
Lead the design and development of enterprise-grade Python applications and distributed systems.
Oversee architecture and implementation of data pipelines, APIs, and large-scale data processing frameworks.
Ensure solutions are designed with high availability, fault tolerance, and observability.
Lead end-to-end engineering ownership for mission-critical database and master data platforms, including development, support, maintenance, lifecycle management, performance, reliability, and operational excellence.
Apply advanced database optimization strategies across Oracle and related platforms, including performance tuning, partitioning, query optimization, resiliency, recoverability, and scalability.
Ensure efficient data modeling, storage, and access patterns across platforms.
Apply hands-on expertise in Oracle/ODI, Python, Spark, AWS EMR, Redshift, S3, and Iceberg to design, build, and modernize enterprise data platforms.
Lead transition of legacy and operational master data capabilities toward modern cloud data lakehouse architecture using S3, Iceberg, Redshift, and medallion patterns for ingestion, transformation, curation, quality, and governed consumption.
Architect and standardize CI/CD pipelines using Jenkins and modern DevOps practices.
Drive adoption of automation-first principles across build, test, and deployment workflows.
Promote DevSecOps best practices and governance controls.
Lead cloud modernization of enterprise master data capabilities, including transition to cloud data lakehouse architecture and medallion-based bronze, silver, and gold data layers.
Influence AWS-based data architecture decisions, including appropriate use of Spark, AWS EMR, cloud storage, orchestration, data quality controls, and scalable consumption patterns.
Drive modernization while preserving operational continuity, enterprise availability, data trust, cost discipline, security, and compliance expectations.
Establish frameworks for performance engineering, observability, monitoring, SLAs, SLOs, SLIs, and operational readiness for enterprise-wide master data platforms.
Lead root cause analysis of critical production issues and define systemic improvements across platform reliability, data quality, resiliency, recovery, and downstream dependency management.
Ensure the platform meets enterprise-grade resiliency, recovery, security, and availability requirements for mission-critical data distribution.
Demonstrate proficiency in using AI tools and AI-assisted engineering practices to improve individual and team efficiency, productivity, solution quality, and delivery velocity.
Use AI responsibly to accelerate analysis, design, coding, testing, documentation, troubleshooting, and operational support while maintaining appropriate engineering judgment, security, and governance standards.
Identify opportunities to apply AI to develop better technical solutions, reduce manual effort, improve platform reliability, and increase the speed of modernization initiatives.
Mentor senior engineers and leads, raising the overall technical bar of the organization.
Drive knowledge sharing, standards adoption, and engineering excellence initiatives.
Serve as a role model for engineering best practices and problem-solving.