Lead Site Reliability Engineer

GM Financial•Arlington, TX

4d•Hybrid

About The Position

Why GMF Technology? Innovation isn’t just a talking point at GM Financial, it’s how we operate. From generative AI and cloud-native technologies to peer-led learning and hackathons, our tech teams are building real solutions that make a difference. We’re committed to AI-powered transformation, using advanced machine learning and automation to help us reimagine customer interactions and modernize operations, positioning GM Financial as a leader in digital innovation within a dynamic industry. Join us and discover a workplace where your ideas matter, your development is prioritized, and you can truly make a global impact. About the Role: We are expanding our efforts into complementary data technologies for decision support in areas of ingesting and processing large data sets. Our interests are in enabling data science and search based applications on large and low latent data sets in both a batch and streaming context for processing. To that end, this role will incorporates aspects of software engineering and operations, combining SRE and DevOps skills to come up with efficient ways of managing and operating applications. The role will require a high level of responsibility and accountability to deliver technical solutions. The data sets we deal with support both off-line and in-line machine learning training and model execution. Other data sets support search engine based analytics. Exploration and deployment of technologies activities include identifying opportunities that impact business strategy, selecting data solutions software, and defining hardware requirements based on business requirements. Responsibility also includes documentation of procedures for deployment, monitoring, managing and switching the environments in production and disaster recovery sites. This role participates along with team counterparts to architect an end-to-end framework developed on a group of core data technologies

Requirements

Excellent knowledge of Kubernetes Administration, Deployments & Upgrades
Excellent Knowledge on Apache Spark administration on various platforms
Strong working knowledge of Object Store(MinIO) and Spark cluster security, networking connectivity and IO throughput along with other factors that affect distributed system performance
Strong working knowledge of disaster recovery, incident management, and security best practices
Working knowledge of containers (e.g., docker) and major orchestrators (e.g., Mesos, Kubernetes, Docker Datacenter)
Working knowledge of software defined networking
Working knowledge of hardening Data at Rest with key based encryption technologies
Working knowledge of setting up and customize interactive data analytics tools (e.g., Apache Zeppelin, Jupyter notebooks)
Excellent knowledge on building the docker images to provide Containers-as-a-service
Working knowledge on Azure Administration, Azure DevOps & Azure Kubernetes Service (AKS)
Working knowledge of Pipeline Automation: Azure DevOps (YAML, ARM), Terraform, Jenkins, Chef/Puppet, Ansible
Working knowledge of CICD methodologies like Artifactory/Git/Gitops/Jenkins
Working knowledge of Code Scanning tools: SonarQube, Checkmarx/Blackduck/Twistlock
Working knowledge of Object Storage like S3/MinIO, Bucket policies and administration
Working knowledge of Kubernetes Storage protocols
Experienced with networking infrastructure including VLAN and firewalls
Working knowledge of hardening Kubernetes clusters with network policies like Calico/Tigera, service meshes like Istio, Internal & external load balancers
Proven track record with Red Hat Enterprise Linux & Kubernetes administration
Proficiency in a high-level language like Python, Go, Ruby and/or Java
Solid experience in High Availability and distributed systems, Linux , Data and SAN Storage Networks, NAS and Networking, leveraging tools to instrument and automate proactively and eventually predictive availability solutions
Proven track record leading complex enterprise production support efforts adhering to a mix of DevOps & SRE frameworks
Experience transitioning platforms to the cloud, with knowledge of cloud frameworks & design patterns, micro-service architectures
Extensive Knowledge of networking, including DNS, DHCP, firewalls, load balancers and IP routing
Experience in Monitoring tools - Splunk, Zenoss, Elastic, Appdynamics, Dynatrace, Grafana, Promotheus, Kiali etc.
Ability to grasp difficult concepts, large architectures, and sophisticated designs quickly and troubleshoot with debugging skills across a variety of integrated platforms
Proven capability to provide operational visibility on environment health to Senior Leadership, Technology and Business partners
Receptive, approachable teammate, with the ability to positively interact with business partners, technology teams, offshore, and professional services
Strong customer advocate with excellent written and verbal communication skills
5-7 years of hands-on experience with supporting Linux production environments required
5-7 years of hands-on administration experience on Spark required
3-5 years hands-on experience with scripting with bash, perl, ruby, or python required
3-5 years experience with Docker Datacenter required
2-4 years of hands-on administration experience on Machine learning platforms required
Minimum of 1 year of experience in Mesos, Kubernetes, OpenShift and/or Deis or other such container/platform-as-a-service orchestrator required
Minimum of 1 year of hands-on experience on CICD tools & Technologies required
Minimum of 1 year of lead experience of site reliability engineering team required
Hands-on experience in cloud technologies with Microsoft Azure required
High School Diploma or equivalent required
Bachelor’s Degree in related field or equivalent experience required

Nice To Haves

Master’s Degree Preferred

Responsibilities

Manage/Administer/Deploy Kubernetes and Spark cluster environments, on bare-metal and container infrastructure, including service allocation and configuration for the cluster, capacity planning, performance tuning, and ongoing monitoring
Define and refine processes and procedures for the site reliability engineering practice
Setup, manage and maintain Kubernetes based scalable environments for high-availability and work with vendors for smooth and continuous operations
Work closely with data scientists, data architects, data engineers, ETL developers, cybersecurity, network, Linux, other IT counterparts, and business partners to design and setup the environments to manage the ingested and processed datasets from the external sources, internal systems, and the data warehouse to extract features of interest
Evaluate, research, experiment with data processing, management and scalability technologies in a lab to keep pace with industry innovation while assessing business impact and viability for use cases associated with efforts in hand
Design, setup, test, deploy, monitor, document, and troubleshoot data processing and associated automation issues from the operations perspective
Work with IT Operations and Information Security Operations with monitoring and troubleshooting of incidents to maintain service levels
Work with Information Security Vulnerability Management and vendors to remediate known impacting vulnerabilities
Contribute to the evolving distributed systems architecture to meet changing requirements for scaling, reliability, performance, manageability, and cost
Report utilization and performance metrics to user communities
Contribute to planning and implementation of new/upgraded hardware and software releases
Responsible for monitoring the Linux, Kubernetes, Object Storage(MinIO), Feature Store, and Spark
Research and recommend innovative, and where possible, automated approaches for administration tasks
Identify approaches to efficiencies in resource utilization, provide economies of scale, and simplify support issues
Responsible for administration of Machine Learning platforms & Operations (MLOps) Such as Kubeflow/Jupyterhub/Python
This role will support GMF international operations and will closely align with our GMF IT NorthStar architecture and operating Principles