Senior Site Reliability Engineer (Remote USA)

TechInsights • Quinte West, ON

1d • Remote

About The Position

TechInsights is building the reliability and AI operations foundation for its next chapter — an AI-first intelligence platform that runs the most demanding semiconductor intelligence workflows in the world. We're looking for a Senior Site Reliability Engineer who wants to own that foundation. This is a senior individual contributor role at the technical leadership tier of our Site Reliability Engineering team. You'll own strategic reliability initiatives end-to-end: setting technical direction, defining SLOs and error budgets across our production platform, designing reliability patterns for the AI agent pipelines that power our platform's AI-first capabilities, and enabling our development and AI Engineering teams to build and ship with confidence. What sets this role apart is its scope. You're not just keeping the lights on — you're building the observability, Internal Developer Platform (IDP), and service catalog that a fast-scaling AI platform needs from day one. You'll be the reliability voice in architectural decisions, the engineer who closes the loop between agent failure modes and platform resilience, and the mentor who builds the team's capability rather than their own indispensability. If you have deep SRE experience and want to apply it to AI workloads — agent loop observability, blast radius management, LLM infrastructure reliability — this is the role where that expertise becomes a differentiator. This role is a remote role for candidates based in the United States.

Requirements

• Bachelor's degree in Computer Science, Engineering, or equivalent combination of education and experience
• 6–8 years of progressive experience in site reliability engineering, platform engineering, or DevOps, with demonstrated technical leadership at the senior individual contributor level
• Deep expertise in AWS (EKS, Lambda, CloudWatch, AWS Config) and multi-region architecture patterns
• Proficiency with Terraform and GitOps; experience with policy-as-code (Sentinel, OPA/Rego, or equivalent)
• Hands-on Datadog experience at operational depth: dashboards, SLO tracking, alerting, log management, distributed tracing
• Strong containerization expertise: Docker, Kubernetes (EKS preferred)
• Proficiency in Python and/or Bash; experience building operational tooling; solid understanding of Java and Spring Boot microservice architecture sufficient to make reliability and deployment decisions for EKS-hosted services
• Deep expertise in CI/CD pipeline design and optimization using Bitbucket Pipelines and GitHub Actions
• Familiarity with IDP tooling (Backstage, Atlassian Compass, or equivalent) strongly preferred
• Experience with AI/ML workload infrastructure, LLM API integration, or agentic system operations considered a strong asset

Nice To Haves

• Experience designing reliability architecture for agentic AI systems: agent loop observability, blast radius isolation, graceful degradation for LLM-dependent services
• AWS certifications: Solutions Architect Professional, DevOps Engineer Professional, or equivalent
• FinOps Certified Practitioner or demonstrated cloud cost management experience at scale
• IDP implementation or developer experience program leadership
• Experience in semiconductor, SaaS, or data-intensive platform environments
• Experience operating in environments with export-controlled or regulated data
• Knowledge of BCP/DR program management and formal recovery testing

Responsibilities

• Own SLOs, SLIs, and error budgets for all production services; drive error budget discipline across engineering
• Design reliability patterns for AI agent pipelines: LLM observability, tool-use tracking, failure detection, and graceful degradation
• Architect for blast radius containment — agent failures must have bounded customer impact through isolation, circuit breaking, and rapid recovery
• Mature our Canada Central/West active-active architecture toward 24-hour RTO with full regional failover
• Lead incident response and post-incident reviews that produce durable fixes; maintain DR procedures through regular testing
• Serve as the primary reliability liaison to Software and AI Engineering, translating requirements into actionable standards
• Partner with AI Engineering on compute provisioning, model serving, inference latency, and workload isolation
• Own CI/CD pipeline strategy (Bitbucket Pipelines, GitHub Actions) — set standards, optimize deployment frequency, and ensure teams can ship confidently
• Drive IDP adoption and enable teams on SRE practices: on-call readiness, SLO definition, runbook development, and self-service tooling
• Represent reliability in architectural discussions; surface risk before it's committed to design
• Own the service catalog — a living inventory of all services, AI agents, dependencies, ownership, and SLOs
• Operate Datadog as the single pane of glass for service health, infrastructure, and agentic pipeline telemetry
• Extend observability to AI workloads: LLM latency, token consumption, agent completion rates, and pipeline throughput
• Build golden path templates in Backstage and/or Atlassian Compass so teams ship reliably without routine SRE involvement
• Apply AIOps in Datadog to automate anomaly detection, incident triage, and remediation recommendations
• Own infrastructure as code via Terraform and GitOps; enforce IaC policy in partnership with Trust Assurance
• Own FinOps visibility into AWS cost segments; model cloud cost impact as AI/ML workloads scale
• Formally mentor junior and intermediate SRE engineers, with accountability for their technical growth and career progression
• Build AI-assisted automation to progressively reduce toil and scale the team's operational capacity

Benefits

• Company-sponsored training and development opportunities
• Comprehensive benefits package (health, dental, vision, wellness, 401K Matching, annual fitness reimbursement)
• Flexible vacation policy
• Community involvement opportunities through charitable alliances
• Wellness resources and support
• Inclusive environment that prioritizes diversity, equity, and accessibility
• High-growth company driven by high performance

What This Job Offers

• Job Type: Full-time
• Career Level: Senior