Senior Site Reliability Engineer, Observability

Ripple • Chicago, IL

1d • $160,000 - $200,000 • Hybrid

About The Position

This is an engineering-first role with a coaching dimension. You will spend the majority of your time doing hands-on observability and reliability engineering work: building instrumentation, designing alert configurations, authoring Terraform, and troubleshooting production systems. Alongside that, you will coach and consult with stream-aligned product teams, helping them build operational maturity over time. You will join Ripple’s Technical Operations team and work across Azure (80%) and AWS (20%) environments supporting infrastructure that is predominantly Windows-based (80%), handling significant payment volume for enterprise treasury customers. The incident management program you will help build is early-stage—you will be establishing practices, not inheriting a mature playbook.

Requirements

• 7+ years in Site Reliability Engineering, DevOps, or Platform Engineering with a strong focus on observability and production operations.
• Proven ability to deliver hands-on engineering work while coaching and mentoring teams—comfortable switching between builder and consultant modes.
• Experience working in Agile/Scrum environments and collaborating effectively with cross-functional teams.
• Expert-level hands-on experience with New Relic (APM, Infrastructure, Logs, Synthetics, Alerts) and strong NRQL proficiency for troubleshooting and analysis.
• Deep understanding of structured logging, metrics collection (RED/USE methods), distributed tracing, and designing effective dashboards and alerts.
• Expertise defining and implementing SLOs/SLIs and error budgets for reliability management.
• Hands-on experience with incident management platforms (Incident.IO, PagerDuty, OpsGenie, or similar).
• Experience designing incident response workflows, on-call rotations, escalation policies, and facilitating post-incident reviews that drive actionable improvements.
• Demonstrated ability to troubleshoot complex production issues using observability data across distributed systems.
• Strong Terraform experience: developing and maintaining IaC for cloud infrastructure and monitoring resources; familiarity with IaC governance patterns.
• Proficiency with PowerShell scripting (required given the 80% Windows environment).
• Strong experience with Azure cloud (App Services, Virtual Machines, Azure SQL, networking, monitoring) and working knowledge of AWS.
• Experience with Azure DevOps for CI/CD pipeline authoring and troubleshooting.
• Experience with Octopus Deploy for deployment management and release orchestration.
• Comfort working across both Windows and Linux server environments.
• Familiarity with Slack for operational workflows, alert routing, and incident communication.

Nice To Haves

• Experience with alert noise reduction strategies and observability cost optimization (log ingestion, pipeline rules, cardinality management).
• Background facilitating chaos engineering, game day exercises, or failure injection to build team resilience.
• Knowledge of VM-hosted SQL Server monitoring and performance optimization.
• Familiarity with FinTech compliance requirements (SOC 2, ISO 27001) and audit evidence collection.
• Experience measuring and improving key reliability metrics (MTTR, MTTD, availability, error budgets) at an organizational level.
• Python or Bash scripting experience in addition to PowerShell.
• Familiarity with Jira for incident tracking and workflow automation.

Responsibilities

• Design and implement monitoring, alerting, and dashboards in New Relic (APM, Infrastructure, Logs, Synthetics) across Azure and AWS; write NRQL queries for troubleshooting, analysis, and reporting.
• Define and implement SLOs/SLIs and error budgets; coach teams on using them to balance feature velocity with reliability and communicate system health to stakeholders.
• Lead alert noise reduction and signal quality engineering—tune thresholds, eliminate false positives, and ensure every alert is actionable.
• Optimize observability costs through log ingestion management, pipeline rules, and New Relic configuration governance.
• Partner with engineering teams to improve observability maturity: structured logging, metrics instrumentation (RED/USE methods), distributed tracing, and effective dashboard patterns.
• Develop and maintain Terraform infrastructure as code for provisioning and managing monitoring resources, alert configurations, and observability infrastructure.
• Establish and enforce IaC governance standards for observability infrastructure across teams, providing a repeatable, auditable model for how monitoring resources are managed.
• Author and troubleshoot Azure DevOps pipelines; support teams with deployment visibility, change tracking, and release hygiene as it relates to production reliability.
• Administer and configure Incident.IO: alert routing, notification workflows, Slack and OpsGenie integration, and runbook management—operationalizing what exists today and expanding from there.
• Build out incident management foundations that are largely yours to establish: PIR/postmortem processes, on-call rotation design, escalation policies, incident severity classification, and response playbooks.
• Track and report on MTTR, MTTD, and incident frequency; identify trends and drive continuous improvement in partnership with engineering teams.
• Respond to and debrief on production incidents—providing real-time troubleshooting support and facilitating structured post-incident reviews.
• Enable stream-aligned engineering teams to adopt improved observability and incident management practices through workshops, consultation, and hands-on guidance.
• Collaborate with the Subsystems Platform Team to translate common needs into self-service observability and incident management capabilities.
• Build lasting team competency through documentation, training materials, and knowledge-sharing sessions that outlast any individual engagement.

Benefits

• Professional development budget to support other modes of learning.
• Competitive salary, bonuses, and equity
• Competitive benefits that cover physical and mental healthcare, retirement, family forming, and family support
• Employee giving match
• Mobile phone stipend
• R&R days so you can rest and recharge
• Generous wellness reimbursement and weekly onsite & virtual programming
• Generous vacation policy - work with your manager to take time off when you need it
• Industry-leading parental leave policies.
• Family planning benefits.
• Catered lunches, fully-stocked kitchens with premium snacks/beverages, and plenty of fun events

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed