About The Position

We are seeking an experienced Site Reliability Engineer to join our IT Infrastructure Support team, responsible for ensuring the reliability, scalability, and performance of critical physical security infrastructure and supporting systems. In this role, you will combine software engineering expertise with operations knowledge to build and maintain automation tools, monitoring systems, and processes that support enterprise-grade server, network, and security device management. You will work closely with cross-functional teams to define and enforce service level objectives, reduce operational toil through automation, and drive continuous improvement in system resilience. This position requires 24x5 availability with on-call rotation to ensure uninterrupted support for mission-critical infrastructure.

Requirements

  • 6+ years of experience in Site Reliability Engineering, DevOps, or Infrastructure Engineering
  • Strong proficiency in Python, Bash, and PowerShell for automation scripting, with experience in Go for building high-performance backend services and APIs.
  • Hands-on experience with Infrastructure-as-Code tools (Terraform, Ansible, Chef, or Puppet) and configuration management practices, including drift detection, version control, and automated remediation.
  • Advanced knowledge of Linux and Windows server environments, including Tier 3 troubleshooting capabilities, system hardening, and enterprise-scale server management.
  • Solid understanding of enterprise networking concepts, Cisco device administration, network automation protocols (NETCONF/RESTCONF), and experience with network monitoring and flow analysis tools.
  • Experience implementing and managing monitoring solutions (Prometheus, Grafana, Datadog) and centralized logging platforms (ELK Stack), with ability to create custom dashboards and alerting rules.
  • Proficiency in implementing CI/CD pipelines, automated testing frameworks, and deployment strategies using modern DevOps tooling, with strong emphasis on code quality, security, and maintainability.

Responsibilities

  • Partner with leadership to establish, monitor, and enforce Service Level Indicators (SLIs) and Service Level Objectives (SLOs) for infrastructure tooling, including configuration compliance rates, patch success rates, and deployment latency metrics.
  • Provide Level 3 expertise for tooling-specific incidents, focusing on automating incident remediation workflows and reducing Mean Time To Repair (MTTR) through intelligent automation and runbook development.
  • Identify and automate repetitive manual tasks across managed infrastructure, targeting measurable reductions in operational overhead (e.g., 50% reduction in manual server build time) through scripting and workflow automation.
  • Conduct thorough root cause analysis and lead blameless postmortems for all major service- impacting incidents, driving systemic improvements in tooling reliability and infrastructure resilience.
  • Engineer and maintain automated processes and scripts to populate, update, and synchronize asset management platforms (e.g., NetBox), configuration management databases, and monitoring systems for internal and external stakeholders.
  • Design, develop, and deploy full-stack applications, custom plugins, and automation scripts to extend functionality of management and monitoring systems, enabling direct device interaction for configuration management.
  • Develop and maintain fully automated Infrastructure-as-Code configurations for Windows and Linux server roles using tools such as Ansible, Terraform, or Puppet, including drift detection and auto-remediation capabilities.
  • Build end-to-end automation pipelines for vulnerability patching, security baseline enforcement (CIS benchmarks), and continuous compliance auditing against internal and regulatory standards for physical security devices.
  • Develop API-driven tools for network configuration management, automated firmware updates, pre/post-change validation, and real-time network health monitoring across the device fleet.
  • Deploy and standardize monitoring agents, centralized log collection systems, and custom dashboards with alerts based on critical SLIs (latency, error rate, saturation, traffic) for servers and edge devices.
  • Build automation scripts for intelligent ticket handling, problem validation, and escalation workflows within enterprise ticketing systems, ensuring 2-hour initial response SLAs are consistently met.
  • Participate in 24x5 on-call rotation to provide timely support for infrastructure systems, security devices, and related tooling, ensuring service continuity and rapid incident response.

Benefits

  • Medical provided through UHC (PPO, HSA, Surest options) / Medical provided through Kaiser (HMO option only) for California employees only
  • Dental provided through UHC Nationwide
  • Vision provided by UHC
  • Flexible Spending Account for Health & Dependent Care
  • Pre-Tax Account for Commuter Benefit/Parking & Transit (location-specific)
  • Continuing Education and Professional Development via various integrated platforms, e.g. Udemy and Coursera
  • Corporate Wellness Program provided by Goomi Group
  • Employee Assistance Program
  • Wellness Days
  • 401k Plan
  • Basic and Supplemental Life Insurance
  • Short Term & Long Term Disability
  • Critical Illness, Critical Hospital, and Voluntary Accident Insurance
  • Tuition Reimbursement (available 6 months after start date, capped)
  • Paid Time Off (accrued and prorated, maximum of 120 hours annually)
  • Paid Holidays
  • Any other statutory leaves, paid time, or other ancillary benefits required under state and federal law
© 2024 Teal Labs, Inc
Privacy PolicyTerms of Service