Senior Software Engineer, Backend (Workload Safety)

Affirm

1d • $195,000 - $255,000 • Remote

About The Position

Affirm’s engineering team builds large-scale, highly available global infrastructure shared across multiple financial products. The Workload Safety team develops and enforces standards for configuring and running workloads across Affirm’s growing fleet of more than 85 Amazon EKS clusters, helping maintain availability and latency SLOs. The team identifies relevant industry practices, drives adoption across engineering, and uses policy and automation to make workloads compliant by default.

As a senior engineer, you will help set the team’s roadmap, prioritize workload standards, and establish a repeatable process for taking standards from research through fleet-wide enforcement. You will work with teams operating services, batch jobs, and ML workloads, as well as deployment, capacity, and SRE teams. The position participates in on-call for Kubernetes runtime issues across the fleet.

Affirm is remote-first, and most roles can be performed from almost anywhere within the country of employment. Some positions may occasionally require in-person work at an Affirm office; all new hires are invited to attend an in-person onboarding experience.

Requirements

• At least 5 years of experience running Kubernetes in production at scale; Amazon EKS across many clusters is preferred.
• Deep understanding of Kubernetes resource management and scheduling, including requests and limits, OOM and CPU throttling behavior, affinity and anti-affinity, taints and tolerations, and topology spread constraints.
• Understanding of pod lifecycle and disruption management, including graceful termination, pod disruption budgets, and node draining.
• Experience enforcing configuration standards through admission control and policy-as-code, such as OPA Gatekeeper or Kyverno.
• Proficiency in Go or Python, including building production infrastructure software such as Kubernetes controllers, operators, or admission webhooks.
• Strong production debugging skills and demonstrated ability to turn recurring incidents into systemic fixes.
• Track record of leading technical initiatives across multiple teams, driving adoption of shared standards, and mentoring engineers.
• Experience quantifying infrastructure changes’ impact on availability, latency, and cost.
• Strong written communication skills, including producing standards and guidance teams can adopt independently.
• Equivalent practical experience or a bachelor’s degree in a related field.

Responsibilities

• Define and evolve workload safety standards for Affirm’s EKS fleet, including resource settings, workload isolation and placement, disruption tolerance, and graceful termination.
• Own standards end to end, from research and prototyping through early-adopter onboarding, self-service guidance, automation, and policy guardrails.
• Lead cross-team initiatives from design through fleet-wide rollout, mentor engineers, and set a high bar for design, code review, and operational readiness.
• Isolate workloads by criticality and resource profile to prevent misbehaving workloads from degrading critical paths such as checkout.
• Resolve stuck workloads and node lifecycle problems, including pods ignoring termination signals, pod disruption budgets blocking node cycling, and Karpenter timeouts affecting bin packing and excess-capacity removal.
• Determine and maintain safe cluster operating limits, including pod count, node count, and API server request volume, and develop control-plane scaling patterns for bursty workloads.
• Analyze fleet-wide CPU, memory, disk, and network profiles to guide EC2 instance type strategy, and automate that analysis as workloads change.
• Measure standards’ effects on availability, latency, risk, and cost, and use results to prioritize the roadmap.
• Participate in on-call for Kubernetes runtime issues and turn recurring failure patterns into roadmap items.
• Collaborate with teams operating services, batch jobs, and ML workloads on EKS, and with deployment, capacity, and SRE teams.
• Track Kubernetes community best practices and adapt them to Affirm’s needs.

Benefits

• Health coverage with 100% of premiums covered for employees and dependents, including medical, dental, and vision coverage.
• Monthly stipends for technology setup and health and wellness options.
• Flexible time off and generous holiday calendars.
• Employee stock purchase plan (ESPP) allowing employees to buy Affirm stock at a discount.
• Equity rewards may be included as part of total compensation.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior