Senior Reliability Developer
About The Position
Responsible for changes and updates to public/private cloud infrastructure. Owns all monitors that support AWN business services and our customers. Implements effective automate using programming languages such as Bash, Python, and JavaScript. Ensure our full infrastructure stack is resilient with day-to-day care and feeding. Write and manage Terraform modules and configuration trees for deployments into AWS and OpenStack. Manage automated patching solutions for instances deployed across multiple regions to ensure compliance with strict security requirements. Monitor internal and external TLS certificates for expiration, renewals, and re-deployment into their respective environments. Create and improve service monitoring solutions using Prometheus, Grafana, Zabbix, and various Prometheus exporters (e.g., postgres, node-exporter). Migrate monitoring for legacy production services to a new monitoring system and recreated existing monitoring within Prometheus + Grafana. Collaborate frequently with development teams to implement new features, fixes, or troubleshoot services in lab/production environments. Troubleshoot problems with microservices running within containers, administer containers in Kubernetes clusters, manage cloud components deployed in multiple AWS regions, and maintain CI/CD pipelines. Manage the Cylance AWS Amazon cloud platform through regular administration of Kubernetes cluster upgrades, cluster module upgrades, logging configurations, and resizing of EBS volumes and instances. Troubleshoot issues between interconnected services by working with various teams and resolving issues related to API calls and service unavailability. Create, improve, and maintain detailed documentation for MOPs, service architecture, runbooks, monitoring configurations, and other general resources. Plan and execute service decommissioning in both lab and production environments upon customer or service owner request. Work on an on-call rotation to support business-critical services outside of business hours, respond to escalations, and conduct root cause analysis during incident reviews. Telecommuting permissible from any location within US.
Requirements
- Utilizing Infrastructure-as-Code (IaC) tools including Terraform and configuration management tools including Puppet and Chef to design, implement, and manage multi-region cloud infrastructure deployments across AWS and OpenStack environments.
- Utilizing containerization and orchestration technologies including Docker, Kubernetes, ECS, and EKS to deploy, manage, and troubleshoot microservices architectures in production environments.
- Utilizing Python, Bash, JavaScript, Groovy, and PowerShell scripting languages for automation of infrastructure provisioning, certificate lifecycle management, and automated patching solutions.
- Utilizing monitoring and observability tools including Prometheus, Grafana, Zabbix, AlertManager, and PagerDuty to create dashboards, configure alerts, and implement service health monitoring solutions.
- Utilizing AWS cloud services including EC2, ECS, EKS, ELB, S3, RDS, IAM, Lambda, CloudFormation, VPC, Route53, and CloudWatch to architect, deploy, and manage highly available cloud-based systems.
- Utilizing CI/CD pipeline tools including Jenkins, GitLab, GitHub, Bitbucket, and Gaia to automate testing, deployment processes, and maintain continuous integration workflows.
- Managing Kubernetes cluster operations including cluster upgrades, module upgrades, logging configurations, node scaling, EBS volume management, and pod orchestration across multiple AWS regions.
- Utilizing certificate management and security tools including Keeper for secret management, TLS certificate monitoring, renewal coordination, and automated deployment across production and development environments.
- Troubleshooting distributed systems and microservices communication issues including API failures, service mesh architectures, container networking, inter-service dependencies, and deployment failures.
- Creating and maintaining technical documentation using Atlassian tools including Jira, Confluence, Backstage, and LucidChart for MOPs (Method of Procedures), runbooks, service architecture diagrams, and monitoring configurations.
Responsibilities
- Responsible for changes and updates to public/private cloud infrastructure.
- Owns all monitors that support AWN business services and our customers.
- Implements effective automate using programming languages such as Bash, Python, and JavaScript.
- Ensure our full infrastructure stack is resilient with day-to-day care and feeding.
- Write and manage Terraform modules and configuration trees for deployments into AWS and OpenStack.
- Manage automated patching solutions for instances deployed across multiple regions to ensure compliance with strict security requirements.
- Monitor internal and external TLS certificates for expiration, renewals, and re-deployment into their respective environments.
- Create and improve service monitoring solutions using Prometheus, Grafana, Zabbix, and various Prometheus exporters (e.g., postgres, node-exporter).
- Migrate monitoring for legacy production services to a new monitoring system and recreated existing monitoring within Prometheus + Grafana.
- Collaborate frequently with development teams to implement new features, fixes, or troubleshoot services in lab/production environments.
- Troubleshoot problems with microservices running within containers, administer containers in Kubernetes clusters, manage cloud components deployed in multiple AWS regions, and maintain CI/CD pipelines.
- Manage the Cylance AWS Amazon cloud platform through regular administration of Kubernetes cluster upgrades, cluster module upgrades, logging configurations, and resizing of EBS volumes and instances.
- Troubleshoot issues between interconnected services by working with various teams and resolving issues related to API calls and service unavailability.
- Create, improve, and maintain detailed documentation for MOPs, service architecture, runbooks, monitoring configurations, and other general resources.
- Plan and execute service decommissioning in both lab and production environments upon customer or service owner request.
- Work on an on-call rotation to support business-critical services outside of business hours, respond to escalations, and conduct root cause analysis during incident reviews.
Benefits
- Equity for all employees
- Flexible time off and paid volunteer days
- RRSP and 401k match
- Training and career development programs
- Comprehensive private benefits plan including medical, mental health, dental, disability, life and AD&D, and value-added services
- Robust Employee Assistance Program (EAP) with mental health services
- Fertility support and paid parental leave
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
