Sr. Reliability Engineer

Hims & Hers • ,

20h • $140,000 - $165,000 • Remote

About The Position

We're looking for a Senior Reliability Engineer to make Hims & Hers systems measurably more reliable: building the observability, tooling, and automation that catch problems before people do. Recent work at this level includes preparing our stack for 32x baseline traffic during high-stakes seasonal surges with zero customer-facing issues and improved P95 latency, turning databases into a paved platform with consistent observability and guardrails, and building AI agents that pull FireHydrant, Datadog, Jira, and Confluence into a single incident picture. We use AI first: every engineer gets a Claude Enterprise license, and we expect you to use it as a core part of how you investigate, build, and write, not as an afterthought.

Requirements

• 5+ years as a Software, SRE, Platform, or Infrastructure Engineer, with a track record of owning reliability outcomes for production systems that customers depend on.
• Strong software engineering fundamentals. You solve reliability problems by writing code and building tooling, and you're comfortable reading application code across the stack to find the real cause.
• Hands-on depth in observability and SLO engineering: golden signals, burn-rate alerting, journey-level monitors, and turning noisy alert streams into actionable pages (Datadog preferred; Prometheus/Grafana and OpenTelemetry welcome).
• Production experience with AWS, Kubernetes/EKS, Terraform, and PostgreSQL (RDS/Aurora).
• Experience running or maturing incident management end to end: on-call design, escalation policies, incident command, blameless post-mortems, and action-item follow-through in a tool like FireHydrant or PagerDuty.
• Daily, practical use of AI coding and analysis tools (Claude, Cursor, or similar) to accelerate investigation, code, documentation, and reporting, and clear judgment about when to trust the output and when to verify it.
• Communication skills to explain risk, tradeoffs, and post-incident learnings to engineers and leadership alike, in writing and in review meetings.

Nice To Haves

• Experience building AI agents or LLM-backed automation for operations: incident triage, RCA generation, alert correlation, or observability data analysis, ideally with MCP or tool-calling integrations against Datadog, FireHydrant, or Jira.
• Load-testing and performance-engineering experience at meaningful scale (k6 or similar), including validating systems to well above expected peak.
• Familiarity with service mesh (Istio) and its observability and traffic-management features.
• Background in a regulated or healthcare environment, where reliability and data handling carry patient-safety and compliance weight.
• Experience designing vendor and partner escalation frameworks with defined severities and response SLAs.

Responsibilities

• Own reliability for Tier 1 customer journeys: Define and instrument SLOs, golden signals, and business-level monitors for the journeys that matter most (checkout, telehealth visits, prescription fulfillment), so that degradation is detected by systems rather than by customers or support tickets. Move our "detected by monitors vs. detected by humans" ratio in the right direction and be able to show it.
• Engineer for peak load and failure: Lead capacity and resilience work for high-stakes events and steady-state growth: load-test design, second-by-second analysis of prior events, database and backend bottleneck investigation, and hardening across caching, GraphQL, VPC capacity, and vendor rate limits. Validate before game day, not during it.
• Build incident response as software: Mature FireHydrant, Datadog, and Jira into one connected pipeline: automated incident and RCA ticket creation, SLO burn-rate and composite alerting with deduplication, Tier 1 alert routing, and enforced post-mortem action tracking. Author and maintain the runbooks and severity standards that make any responder effective on any service.
• Automate operational excellence with AI: Build and operate agents and tooling that reduce manual OE work: OER report generation, RCA drafting, stale action-item detection, monitor and runbook gap detection, and OpenClaw agents wired to Datadog and FireHydrant for first-pass incident triage. Ship these as reusable capabilities, not personal scripts.
• Be the deep-debugging expert, and make teams better at it: Teams own debugging their own services, but you are the person they pull in when a problem crosses boundaries: silent service-to-service failures, anomalous traffic, or regressions that span frontend, API, mesh, and database layers. Bring that depth to the hardest cases yourself, then turn what you learn into runbooks, tooling, and pairing so teams can catch and resolve the next one on their own. Partner with Security and product teams on anomaly detection and response, weighing engineering cost and user impact alongside the benefit of any control.
• Maintain the tooling that informs Operational Excellence reviews: Produce the tooling that powers the metrics and narrative that go into bi-weekly VP-level OE reviews and the monthly cross-engineering OER, and drive the resulting action items to closure.
• Raise the bar for others: Document what you build, onboard teammates to roll it out, and coach engineers across squads on SLOs, blameless post-mortems, and on-call practice.

Benefits

• Competitive salary & equity compensation for full-time roles
• Unlimited PTO, company holidays, and quarterly mental health days
• Comprehensive health benefits including medical, dental & vision, and parental leave
• Employee Stock Purchase Program (ESPP)
• 401k benefits with employer matching contribution
• Offsite team retreats

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed