Director, Enterprise Monitoring

OCC • Chicago, IL

1d • $174,300 - $255,500 • Hybrid

About The Position

The Director, Enterprise Monitoring leads OCC’s enterprise monitoring function and is accountable for the day-to-day management, health, and continuous improvement of the tools and practices used to monitor Production systems. The role aligns monitoring and alerting with reliability standards and priorities set by the Executive Director, SRE, and manages the monitoring team, platforms, and vendor relationships to sustain enterprise coverage.

The Director defines and executes the multiyear monitoring and observability strategy and roadmap; owns monitoring standards, coverage requirements, documentation, and runbooks; and translates reliability standards and SLOs into monitoring, alerting, and dashboard configurations. The position oversees platform operations and resilience, expands coverage across technology domains and hybrid, cloud, and containerized workloads, and ensures monitoring readiness before Production releases or changes.

The role owns alert quality, SLI and SLO-based monitoring, automation, monitoring-as-code practices, and reporting of performance measures and risks. It supports Incident, Problem, and Change processes, including escalation during Production incidents and after-hours coverage under the on-call model defined with the Executive Director, SRE. The Director also owns the monitoring budget, vendor and license management, tool rationalization, remediation of audit findings, and evidence and support for regulatory and audit requests.

The position manages the enterprise monitoring team, including hiring, training, development, performance management, work assignments, performance goals, and succession depth. Team schedules and coverage are coordinated with SRE to support Production availability. After-hours and weekend work may be required during maintenance windows and Production incidents.

Requirements

• Excellent oral and written communication skills, including explaining technical monitoring information to nontechnical audiences, coordinating cross-functional troubleshooting, and leading RCA processes to closure.
• Demonstrated ability to troubleshoot complex Production issues, perform RCA, and drive long-term corrective actions.
• Effective incident management skills and a structured, analytical approach to problem solving.
• Experience partnering with development teams to shape architecture, define SLIs and SLOs, and embed reliability into services from design through operation.
• Experience presenting to senior leadership and governance forums and communicating risk clearly.
• Demonstrated ability to lead, develop, and retain high-performing technical colleagues.
• Demonstrated experience leading tool consolidation, migration, or modernization programs.
• Strong organization and time management skills, with the ability to prioritize in a fast-paced environment.
• Understanding of the urgency of SLAs and the need for timely responses.
• Ability to work in a high-intensity environment, including during Production incidents.
• Sound analytical, diagnostic, and judgment skills.
• Ability to work effectively with technical colleagues, business partners, and vendors.
• Ability to build partnerships across the company and influence peers and colleagues to achieve results.
• Ability to optimize available resources and approach issues pragmatically.
• Deep, hands-on knowledge of enterprise monitoring tools, including infrastructure monitoring, application performance monitoring, log management, and event management platforms.
• Strong understanding of monitoring and observability ecosystems, including Splunk, Dynatrace, Datadog, Prometheus, Logic Monitor, Grafana, ELK/EFK, and OpenTelemetry.
• Hands-on experience with tracing and logging backends such as Tempo/Jaeger, Loki/Elastic/Splunk, CloudWatch, and AWS X-Ray.
• Experience with Kubernetes and Kafka observability, including EKS metrics, kube-state-metrics, consumer lag, and broker health.
• Fluency in the Golden Signals and the RED and USE methods as applied frameworks for monitoring and alerting design.
• Experience designing and tuning alerts, thresholds, dashboards, and metrics.
• Working knowledge of observability concepts, including SLIs, SLOs, and distributed tracing.
• Experience with automation and IaC tools such as Chef, Ansible, and Terraform, and structured data formats such as JSON and YAML, applied to monitoring configuration, onboarding, and reporting.
• Understanding of client-server relationships, network concepts, stack trace analysis (including TCP dumps, heap dumps, CPU and memory analysis, and thread dumps), logging and monitoring tools, high availability, and business continuity planning.
• Understanding of how servers, networks, storage, databases, and applications are monitored.
• Working knowledge of ITIL principles, particularly Event, Incident, and Problem Management.
• Experience with vendor, license, and budget management, including contract negotiation and vendor performance management.
• Bachelor’s degree in Computer Science, Information Technology, or a related discipline, or an equivalent combination of education and work experience.
• 10 or more years of experience in Production systems operations or monitoring, including at least 5 years focused on enterprise monitoring.
• At least 5 years of experience managing technical teams.

Nice To Haves

• Experience with telemetry pipeline tools such as Fluent Bit, Vector, or Cribl for routing, sampling, redaction, and cost control.
• Experience with synthetic monitoring, real user monitoring, and observability based on eBPF.
• Experience with event correlation, anomaly detection, or AIOps capabilities.
• Familiarity with CMDB and service mapping practices.
• Knowledge of the Clearing and Finance industry and its regulatory expectations for operational resilience.
• ITIL certification.
• Splunk, Dynatrace, or AWS Certified Solutions Architect certification.

Responsibilities

• Define and execute the multiyear enterprise monitoring and observability strategy and roadmap, including target architecture, tooling direction, and investment priorities.
• Own monitoring standards, coverage requirements, documentation, and runbooks; recommend changes to reliability standards to the Executive Director, SRE.
• Translate SRE leadership’s reliability standards and SLOs into monitoring, alerting, and dashboard configurations.
• Define monitoring readiness criteria and recommend against proceeding with a Production release or change when criteria are not met.
• Manage the operation, maintenance, and upgrades of enterprise monitoring tools and platforms.
• Ensure monitoring platform availability and resilience, including high availability, disaster recovery, and monitoring of the monitoring tools.
• Extend monitoring across infrastructure, networks, storage, databases, and applications, including hybrid, cloud, and containerized workloads such as Kubernetes and Kafka platforms.
• Maintain accurate service-to-infrastructure mapping with CMDB owners so alerts reflect business service impact and reach the correct support teams.
• Partner with Information Security to support security operations with monitoring and log data and enforce access control and log retention requirements.
• Tune thresholds, reduce false and duplicate alerts, and ensure alerts route to the correct support teams.
• Define SLIs with SRE and development teams and implement SLO-based and business-service-level monitoring, including error budget reporting.
• Introduce event correlation, anomaly detection, and other AIOps capabilities where they reduce noise and speed detection.
• Automate monitoring onboarding, configuration, and reporting to reduce manual effort and improve consistency.
• Advance monitoring as code through IaC practices and CI/CD integrations so new services are monitored by default.
• Maintain monitoring coverage standards and partner with Release Management and Change Management to ensure new and changed applications and infrastructure are monitored before Production.
• Ensure monitoring data is available during incidents and close monitoring gaps identified in post-incident reviews.
• Serve as an escalation point for monitoring tools and alert behavior during Production incidents, including after-hours coverage in line with the on-call model defined with the Executive Director, SRE.
• Provide monitoring data, trend reporting, and metrics to support SRE analysis of Production performance and capacity.
• Define, track, and report monitoring performance measures, including coverage, alert noise, mean time to detect, and the share of incidents detected by monitoring.
• Surface monitoring risks, gaps, tool limitations, and roadmap progress to the Executive Director, SRE, and senior leaders, including risk and governance forums as required.
• Own the monitoring budget, including forecasting, investment business cases, and approval of spend within delegated authority.
• Manage monitoring vendors, licenses, and renewals; negotiate contracts and SLAs; and maintain exit and transition plans for critical tools.
• Identify opportunities to consolidate or rationalize tools and lead resulting migrations and decommissioning.
• Oversee remediation of monitoring findings and observations, ensuring root causes are addressed, residual risk is improved, and remediation is timely.
• Maintain evidence of monitoring controls and support internal and external audit and regulatory requests related to monitoring.
• Support OCC’s compliance with applicable regulatory expectations for system monitoring and operational resilience.
• Adhere to Production security requirements, ITIL processes, and best practices.
• Partner with Production Support, Environment Operations, Platform Engineering, Application Development, Linux, Storage, Networking, Database Engineering, Architecture, and Business Operations to ensure consistent monitoring across technology domains.
• Manage the enterprise monitoring team, including hiring, training, development, performance management, and work assignments.
• Build team capability and succession depth and set performance goals tied to success measures.
• Coordinate team schedules and coverage with SRE to support Production availability requirements.

Benefits

• Hybrid work environment, with up to 2 days per week of remote work.
• Tuition reimbursement.
• Student loan repayment assistance.
• Technology stipend for using a device of choice to connect to the network while working remotely.
• Generous PTO and parental leave.
• 401(k) employer match.
• Competitive medical, dental, and vision benefits.
• Employees may be eligible for a discretionary annual bonus; awards are not guaranteed.

What This Job Offers

• Job Type: Full-time
• Career Level: Director