System Operations Engineer

MSG Entertainment Holdings, LLC•New York, NY

1d•$83,000 - $130,000•Onsite

About The Position

Madison Square Garden Entertainment Corp. (MSG Entertainment) is seeking a hands-on, junior to mid-level System Operations Engineer to run the day-to-day operation of its Azure Virtual Desktop (AVD) environment and provide first-line infrastructure support across the department. AVD is the role’s primary responsibility, including keeping the environment available for users, monitoring system health, provisioning users and resources, updating images, and troubleshooting access, performance, and profile issues. The engineer will also provide Level 1 support and perform scheduled maintenance across Azure, Nutanix, Qumulo, Infinidat, Rubrik, and other infrastructure platforms. Work includes monitoring server, storage, and backup health; responding to alerts; working tickets and service requests; performing basic troubleshooting; and escalating issues beyond documented procedures to senior engineering. The role works with senior engineers and cross-functional teams, supports studio, production, and live event teams, and helps maintain infrastructure documentation and runbooks. The position requires participation in a team-based 24/7 on-call rotation and availability for overnight support and maintenance windows. On-site appearance and presentation guidelines apply. The listed pay range is $83,000–$130,000 USD.

Requirements

4–6 years of related experience.
Bachelor’s degree or an equivalent combination of education and experience.
3+ years of experience working with high-performance systems platforms such as compute, storage, data centers, and automation.
Previous experience supporting enterprise-level infrastructure with absolute fault tolerance in mind.
Integration experience with event correlation platforms and ticketing systems such as ServiceNow.
Ability to troubleshoot server- and storage-related outages using infrastructure monitoring tools such as SolarWinds, Dynatrace, and Grafana.
Self-directed and solutions-oriented, with sound decision-making skills and a hands-on, production-minded approach focused on customer service and operational excellence.
Experience working closely with production, external partners, and creative development teams.
Production experience with Nutanix, Azure, and AWS, including physical, virtual, and VDI server infrastructure.
Storage and backup expertise, including NAS, SAN, LUN, and Rubrik backup platforms.
Expertise in Ansible automation and scripting with Python, PowerShell, and Bash, along with strong Linux and Windows administration skills.
Familiarity with networking, including routing, switching, firewalls, and load balancing.
Must be available for on-call and overnight support, change management, and patching responsibilities on a rotating team schedule.

Responsibilities

Operate the Azure Virtual Desktop (AVD) environment, including host pools, session hosts, and user profiles.
Perform AVD provisioning, user onboarding and offboarding, and routine configuration tasks through Nerdio.
Serve as the first point of contact for VDI issues, resolve common performance, access, and profile problems, and escalate other issues.
Apply scheduled updates to AVD golden images according to established procedures.
Monitor AVD capacity and utilization, flag trends, and make routine scaling adjustments.
Monitor server and storage health across bare metal, Nutanix, and Azure environments; respond to alerts and investigate anomalies.
Perform routine Qumulo and Infinidat storage tasks, including health checks and standard provisioning requests.
Monitor Rubrik backup jobs, resolve common job failures, and perform restores following documented procedures.
Support S3 bucket workflows and routine data backup, archive, and media management tasks.
Perform basic hardware troubleshooting, including component replacement, and assist with vendor support cases and RMAs.
Assist senior engineers with server and storage installs, upgrades, and buildouts.
Provide Level 1 infrastructure support, work incidents and service requests in ServiceNow, document work clearly, and escalate complex or unfamiliar issues.
Execute scheduled firmware, software, and operating system patching across infrastructure systems, including during overnight maintenance windows and according to approved change plans.
Submit change requests for review and carry out approved changes as directed.
Participate in a team-based 24/7 on-call rotation, providing first response and escalating as needed.
Run existing Ansible playbooks and scripts for patching and routine maintenance; script repetitive tasks in PowerShell with senior engineering support.
Use SolarWinds and other monitoring tools for daily health checks and alert response.
Maintain infrastructure documentation, including diagrams, equipment lists, and change records; keep runbooks and procedures current and identify missing or unclear procedures.
Work with senior engineering, network, and application teams to troubleshoot cross-system issues.
Provide hands-on infrastructure assistance to studio, production, and live event teams.
Support Genetec CCTV backend server infrastructure and provide coverage across other departmental areas as needed.
Provide exceptional experiences for guests, partners, and team members, including following on-site appearance and presentation guidelines.