Principal Software Engineer, DevOps

Legion
•$220,000 - $255,000•Remote

About The Position

Legion is hiring a Principal Software Engineer for its site reliability engineering team. The role is responsible for building the tools, infrastructure, and services behind a secure, highly scalable, and cost-effective AWS- and Kubernetes-based cloud platform. Work spans infrastructure tooling and automation, build and deployment pipelines, monitoring and logging, and containerization, supporting the platform that runs Legion’s workforce management solution for customers. The position also involves building and operating production AI agents to reduce SRE toil, including incident triage, runbook execution, alert enrichment, and capacity and cost analysis. The engineer will own agent guardrails, rollback, reliability, and the platform that enables other engineers to build agents safely, including sandboxed execution, scoped credentials, tool and MCP integrations, cost controls, and agent observability. The role supports deployment during and outside regular office hours and participates in an on-call rotation.

Requirements

  • 12 or more years of experience in SRE, DevOps, or other SaaS operations.
  • At least 3 years of hands-on experience managing AWS services and cloud infrastructure using Terraform.
  • At least 3 years of experience with containerized cloud solutions using Docker, Kubernetes, or AWS EKS; familiarity with Helm charts.
  • At least 5 years of experience with at least one programming language such as Go, Python, Bash, or Perl.
  • At least 5 years of experience with one or more of infrastructure automation tools, CI/CD pipelines, and configuration management tools, such as Terraform, Ansible, Git, Jenkins, and Chef.
  • Hands-on experience with one or more Linux/Unix platforms, such as RedHat, CentOS, Ubuntu, or Amazon Linux.
  • Hands-on experience building production agentic AI systems, including tool integrations, evaluation, and failure handling; experience must go beyond using AI coding assistants.
  • Bachelor’s or master’s degree in Computer Science, Engineering, or a related field.
  • Demonstrated ability to work with remote teams.

Nice To Haves

  • AWS certification.
  • A track record of introducing AI into an engineering organization’s SDLC with measurable results, along with the judgment to identify where AI should not be used.
  • At least 3 years of experience with observability tools such as Splunk, Nagios, Elasticsearch, Kibana, CloudWatch, and Logstash, and ways to scale these systems.
  • At least 3 years of experience with AWS RDS or Aurora MySQL.

Responsibilities

  • Support Legion’s public cloud platform using multiple cloud services and containerization.
  • Develop infrastructure automation using Terraform, Chef, Jenkins, and Golang.
  • Create and manage production alerts, respond to alerts, and conduct root-cause investigations.
  • Develop automated operational runbooks.
  • Support deployment of Legion’s workforce management solution during and outside regular office hours.
  • Participate in an on-call rotation.
  • Build and operate production AI agents to reduce SRE toil, including incident triage, runbook execution, alert enrichment, and capacity and cost analysis.
  • Own the guardrails, rollback, and reliability of AI agents.
  • Own the platform that enables engineers to build agents safely, including sandboxed execution, scoped credentials, tool and MCP integrations, cost controls, and agent observability.

Benefits

  • $0 monthly premium and other flexible medical, dental, and vision plans effective on the first day of employment.
  • 401(k) plan.
  • Discretionary paid time off and paid holidays.
  • Parental leave.
  • Stock equity.
  • Monthly wellness reimbursement.
  • Monthly lunch on Legion.
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service