Principal Infrastructure Engineer

Surescripts•Minneapolis, MN

3d•$161,600 - $197,600•Hybrid

About The Position

The Principal Infrastructure Engineer is crucial to ensuring the stability, efficiency and security of the products we operate as crucial components of the nation's healthcare ecosystem. As a member of our Operations team, a Principal Infrastructure Engineer is expected to have knowledge and experience spanning many disciplines with the ability to lead, communicate and execute as a subject matter expert on specific topics or systems. This role may be called upon to troubleshoot deeply technical problems, architect a solution for a new cloud-native application or write an Ansible module to support deployment of new security tooling. At the Principal level, our Platform Engineers are expected to be able to speak on a broad variety of topics and successfully lead projects both internal to our Systems or Operations teams as well as in collaboration with diverse groups of technical and non-technical team members. This position has a specific focus on systems performance tuning and analysis that is becoming even more crucial as we begin operations in Google Cloud Platform. Sizing, tuning troubleshooting and performance monitoring now draw a direct line to the budget as we transition to the minute-by-minute pricing models of the cloud. Knowing where we can gain efficiency and where we cannot sacrifice performance has become a crucial part of our architectural discussion and on-going operational health.

Requirements

Bachelor’s degree in IT, computer science, information sciences or related field or equivalent experience
8+ years of experience in systems engineering role
Experience in a production operation environment with high uptime requirements.
Familiarity with configuration management tools (Ansible preferred).
Knowledge of virtualization and containerization technologies – Vmware (vSphere / ESXi 8x) and Kubernetes preferred
Proficiency in scripting languages (Bash, Powershell, Python, Perl).
Experience with performance, logging and monitoring tools (Zenoss, Splunk, Datadog, Prometheus, Grafana, etc.)
Strong background in system metrics analysis and monitoring.
Experience working in cloud environments (GCP preferred)
Valid U.S. work authorization allowing work without restrictions with Surecripts in the U.S.
Ability to use judgement, withstand moderate amounts of stress and maintain attention to detail with, or without, a reasonable accommodation.

Nice To Haves

Experience operating in an environment with sensitive health data
Experience with RedHat, RockyOS and Windows
Certifications like RHCE or Google Cloud Architect/Engineer
Understanding of network operations, network troubleshooting and the ability to understand the output of packet captures
Experience as a Team Lead, or non-people manager leadership role.

Responsibilities

Participate in development of strategic goals and plans for new and existing systems infrastructure.
Research, design, architect, and implement new technologies and solutions that span on-premise and cloud platforms in a method consistent with our security policies and infrastructure-as-code principals.
Implement performance tuning strategies to optimize system performance, resource utilization, and overall efficiency.
Collaborate with other teams to design and implement scalable and resilient infrastructure solutions.
Assist in support of issues impacting the performance or reliability of production transactions.
Recommend/implement policies, procedures, and technologies to ensure system security through secure system access, monitoring and control.
Participate as necessary in routine security evaluations.
Implement automation solutions for system provisioning, configuration management, and performance tuning.
Ensure compliance with policies, standards, licensing agreements, configuration guidelines, and best-practices for maintaining regulatory compliance.
Ensure all systems and processes are well-documented and comply with internal policies and industry standards.