Lead Performance & Observability Engineer

Intercontinental Exchange Holdings, Inc.•Atlanta, KS

About The Position

Intercontinental Exchange, Inc. (ICE), the owner of the New York Stock Exchange (NYSE), is hiring a results-oriented, self-motivated technical lead for its Clearing Performance Engineering team in Atlanta. The role operates in a cutting-edge technology environment supporting critical financial exchanges and clearinghouses. The successful candidate will drive performance engineering strategy across multiple platforms, mentor peers, and deliver deep-dive analysis of complex, time-sensitive systems. The Lead Performance & Observability Engineer owns end-to-end performance strategy for complex, event-driven Java platforms. The role defines testing methodologies, builds durable test harnesses, leads observability initiatives, and provides authoritative guidance on performance trade-offs across application, JVM, operating system, and hardware layers. It requires close collaboration with software architects, developers, quantitative library owners, infrastructure teams, and project managers to ensure reliability, scalability, and throughput in a mission-critical environment where end-of-day processing windows are measured in minutes. The role also champions AI-augmented performance workflows, while establishing standards for validating AI-generated output before it reaches production.

Requirements

Bachelor's degree or equivalent in Computer Science, Engineering, or a related field.
At least 8 years of experience in performance engineering, performance testing, or Java development in high-volume, low-latency transactional systems.
Deep expertise in JVM internals, including heap-dump analysis, thread-dump analysis, GC-log interpretation, and memory profiling.
Proficiency with Java and scripting languages such as Python, Groovy, or Linux shell for test-harness foundations and automation.
Ability to perform scalability analysis, project system behavior as data volumes grow, and validate linearity assumptions with empirical evidence.
Strong grasp of critical-path analysis, including identifying time-constrained operations, separating them from non-critical work, and validating architectural changes that improve throughput within a fixed time window.
Hands-on experience designing and operating multi-signal observability stacks covering metrics collection, distributed tracing, and continuous profiling across Java application environments.
Proficiency with Prometheus and VictoriaMetrics for metrics collection, storage, and querying, including health-scoring rules, recording rules, and alerting.
Experience building custom metrics exporters for business-layer and application-specific signals.
Proficiency with Grafana for operational and performance dashboards, including programmatic dashboard management, datasource governance, and standards across multiple teams.
Familiarity with distributed tracing and continuous-profiling tools, and ability to use trace and profile data alongside load-test results for root-cause analysis.
Experience building automated post-test reporting pipelines that combine observability data and load-test results into actionable performance narratives.
Experience with event-driven, message-based architectures such as Kafka, IBM MQ, or equivalent, and ability to test and profile throughput and latency across event pipelines.
Experience with load-generation frameworks such as JMeter, Gatling, or custom harnesses, and ability to design workloads representative of production traffic.
Excellent verbal and written communication skills, including presenting findings to technical architects and non-technical stakeholders.
Ability to work across teams with varying levels of performance-domain knowledge and build collaborative relationships with application owners.
Demonstrated proficiency with AI coding and analysis tools such as GitHub Copilot, Claude, Cursor, or equivalent for performance-specific tasks, including generating profiling scripts, writing Prometheus queries, analyzing GC logs, and building test-harness scaffolding.

Responsibilities

Define and own performance engineering strategy across multiple critical platforms, including standards for testing approaches, tooling, and reporting.
Lead deep-dive performance investigations of CPU-bound, multithreaded Java systems at JVM, operating system, and hardware levels.
Analyze scalability as data volume and instrument counts grow, and produce capacity projections to guide architecture decisions.
Lead critical-path segregation analysis, identify time-constrained operations, propose architectural approaches such as head-start strategies or out-of-band processing, and validate their impact.
Design and own an observability framework spanning metrics, distributed tracing, and continuous profiling across platform components.
Build and maintain platform-wide health scoring and alerting using Prometheus and VictoriaMetrics, and drive consistent instrumentation and Grafana visualization standards.
Develop custom metrics exporters for business-layer signals such as database growth, queue depths, and pipeline latency.
Architect and maintain Grafana dashboards and set standards for dashboard organization, datasource governance, and operational visibility across teams.
Lead distributed tracing and continuous profiling initiatives across Java application environments, using trace and profile data alongside load-test findings to accelerate root-cause analysis.
Build automated post-test analysis pipelines correlating load-test results with observability data to produce clear performance narratives for technical and non-technical stakeholders.
Design and build test harnesses that measure version-to-version performance regressions for compute-intensive components.
Tune JVM thread pools, garbage collection, and heap allocation for high-throughput, latency-sensitive processing pipelines.
Design representative production-traffic workloads using load-generation frameworks such as JMeter, Gatling, or custom harnesses.
Create and maintain automation scripts and tooling for repeatable performance analysis.
Serve as a technical liaison between performance, development, infrastructure, and operations teams, translating findings into actionable, evidence-backed recommendations.
Champion AI-augmented workflows for root-cause analysis, test-harness development, and benchmark anomaly detection, while setting standards for validating AI-generated output.