Major Incident Manager

Sonar•Geneva, IL

13h•Onsite

About The Position

Sonar is seeking a Major Incident Manager to join their team. This role is pivotal in automating and enhancing the software development lifecycle, empowering fellow SonarSourcers to deliver with speed, confidence, and security. The Major Incident Manager will use and create automation tools to monitor and observe production infrastructure services, both on-premises and in the cloud. The ideal candidate is allergic to repetitive tasks, preferring to maximize automation and reliability, and is an expert in change management, infrastructure management, system support, and configuration management.

Requirements

Professional experience provisioning and managing complex infrastructure using tools like Terraform or CloudFormation (AWS), or similar tools like Ansible or Puppet for configuration management.
Hands-on experience with a major cloud provider (AWS, GCP, Azure) or managing large-scale internal/private cloud infrastructure.
Practical experience defining, measuring, and reporting on Service Level Indicators (SLIs) and Service Level Objectives (SLOs) for critical services.
Proven experience with modern observability platforms (e.g., Prometheus/Grafana, ELK/EFK stack, proprietary systems, or vendor solutions like Datadog/Splunk) for proactive issue identification.
Strong understanding of core networking concepts (TCP/IP, DNS, Load Balancing, Firewalls, Proxies) sufficient to debug complex service connectivity and latency issues.
Experience implementing security best practices via code, such as automated vulnerability scanning, configuration hardening, secret management (e.g., HashiCorp Vault), and key rotation.
Practical experience managing large-scale IAM systems (e.g., implementing least-privilege policies, single sign-on).
Experience running or significantly contributing to post-incident reviews (post-mortems) and prioritizing resulting engineering work (error budget management).

Responsibilities

Monitoring critical security infrastructure (e.g., identity platforms, firewalls, compliance systems) and core infrastructure components.
Using and maintaining dashboards tied to Service Level Objectives (SLOs), triaging high-severity alerts, and analyzing the current Error Budget burn rate to guide prioritization.
Writing, reviewing, and testing code (e.g., Python, Go, Terraform, or proprietary tools) to automate the deployment, configuration, and security hardening of infrastructure.
Treating infrastructure and security policies as software to ensure consistency and prevent configuration drift.
Identifying, scoping, and implementing automated solutions for manual, repetitive, and time-consuming tasks (toil) related to security patching, compliance checks, certificate rotations, or infrastructure maintenance.
Maintaining and enhancing the DevSecOps security tools integrated into the CI/CD pipelines (e.g., static analysis, vulnerability scanning, security configuration checks).
Ensuring the end-to-end logging, metrics, and tracing (observability) systems for both infrastructure and security tools are robust, accurate, and provide immediate diagnostic capability during incidents.
Participating in the on-call rotation and actively engaging in engineering solutions derived from post-mortems.
Turning incident root causes into preventative measures implemented via code.
Improving runbooks into automated actions.
Reducing Mean Time To Resolution (MTTR) for future incidents.