Lead Principal Core Infrastructure Engineer

Oracle•Nashville, TN

27d•$146,300 - $306,400

About The Position

Mentors teams and leads the architecture of highly scalable, interdependent distributed systems. Identifies and removes performance and scalability bottlenecks for hyper-scale workloads, defines scalability requirements with stakeholders, and designs elastic, high-impact systems while advancing innovation in data plane platforms. Engineers and oversees fault-tolerant, in-service-upgradable designs; optimizes resilience mechanisms such as load-shedding, throttling, and rate-limiting; and sets SLO-aligned durability and availability standards across dependent services. Establishes KPIs and advanced telemetry, applies formal verification for complex features, and develops robust replication and synchronization strategies. Advises on and leads resolution of complex production issues, sets operational readiness and standard operating procedure standards, and directs incident response and root cause analyses. Architects advanced security controls, drives remediation and compliance, and delivers enterprise-level automation, including Infrastructure as Code, and change strategies that enable safe, automated patching, updates, and rollbacks. The role also directs planning and execution of critical, high-impact projects; builds alignment with cross-functional leaders and external stakeholders; serves as an escalation point for complex problems; advances industry knowledge and continuous learning; improves organizational processes; and contributes expertise to talent development and hiring decisions.

Responsibilities

Mentor teams in the architecture and design of highly scalable, interdependent distributed systems, including horizontal and vertical scalability, performance, and distributed state management.
Identify performance and scalability bottlenecks and recommend code or system optimizations for large-scale data processing and high-throughput workloads.
Collaborate with stakeholders to define scalability requirements that meet customer expectations.
Design elastic, high-impact systems and evaluate whether systems meet nonfunctional scalability requirements while anticipating future business needs.
Drive innovation in the use of data plane platforms.
Design and oversee fault-tolerant systems capable of in-service updates through redundancy, replication, and automatic failover.
Lead the design of systems that handle service disruptions, including network partitions, by addressing consistency, availability, and partition tolerance.
Optimize mechanisms for handling network unreliability, including load-shedding, throttling, and rate-limiting.
Design durable interdependent systems that adhere to SLOs and establish availability and durability standards for other computing services.
Define KPIs and telemetry to identify risks, gaps, and cyclical dependencies in running systems.
Create and customize complex dashboards, telemetry systems, and alerting mechanisms to proactively monitor system health and reliability.
Apply industry standards and techniques to verify correctness in interdependent systems, including formal verification of complex features such as with TLA+.
Develop data replication and synchronization strategies that ensure data integrity and availability.
Advise on diagnosis, debugging, and resolution of complex issues in active systems.
Develop strategies to prevent interruptions and eliminate customer maintenance windows when resolving issues.
Maintain expertise in system dependencies, dependents, and owned systems to support troubleshooting and performance.
Set operational readiness and standard operating procedure standards, and hold third-party partners accountable for meeting them.
Oversee operational support rotations, guide incident response, and lead root cause investigations to prevent future occurrences.
Architect security measures for data and applications in multi-tenant environments and lead protection initiatives.
Guide remediation plans for security vulnerabilities.
Ensure cloud infrastructure complies with industry standards and regulations, and guide project documentation.
Develop enterprise-level automation tools and strategies, such as Infrastructure as Code, and oversee implementation.
Align change management plans and organizational initiatives for patching, updating, and rolling back applications; design systems to automate these processes.
Direct timelines, deliverables, and applicable budgets for critical, high-impact projects, adapting plans to changing business priorities.
Influence cross-functional leaders and external stakeholders, build partnerships, and incorporate diverse perspectives.
Lead advanced problem-solving as an escalation point, guide data-driven approaches to ambiguous issues, identify root causes, and implement preventive solutions.
Contribute thought leadership to the organization and industry, monitor emerging trends and standards, and promote continuous learning and knowledge sharing.
Develop and implement process improvements, assess their effectiveness, and create metrics to measure improvement outcomes.
Advise on talent needs and organizational talent strategy; contribute expertise to interviews, candidate assessments, hiring decisions, and talent development.