System Operations Engineer

MSG Entertainment Holdings•New York City, NY

1d•Onsite

About The Position

Madison Square Garden Entertainment Corp. is seeking a hands-on System Operations Engineer to operate its Azure Virtual Desktop (AVD) environment and provide first-line infrastructure support. The role is junior to mid-level and focused on keeping AVD reliable for users, monitoring infrastructure health, responding to tickets and service requests, troubleshooting routine issues, and performing scheduled maintenance across Azure, Nutanix, Qumulo, Infinidat, and Rubrik environments. AVD is the primary day-to-day responsibility, including provisioning, image updates, and user profile support. The engineer will resolve issues covered by documented procedures and escalate complex or unfamiliar issues to senior engineering. The position also supports infrastructure systems and storage, backup, and media workflows; assists with hardware troubleshooting and infrastructure installations; and works with senior engineering, network, and application teams on cross-system issues. The engineer supports studio, production, and live event teams, provides coverage for Genetec CCTV backend server infrastructure, maintains infrastructure documentation and runbooks, and participates in a team-based 24/7 on-call rotation. The role requires availability for overnight support and maintenance windows on a rotating schedule, and adherence to on-site appearance and presentation guidelines.

Requirements

4–6 years of related experience.
Bachelor’s degree or an equivalent combination of education and experience.
At least 3 years of experience working with high-performance systems platforms such as compute, storage, data centers, and automation.
Previous experience at an enterprise-level company supporting infrastructure with absolute fault tolerance in mind.
Experience integrating with event correlation platforms and ticketing systems such as ServiceNow.
Ability to troubleshoot server- and storage-related outages using infrastructure monitoring tools such as SolarWinds, Dynatrace, and Grafana.
Self-directed, solutions-oriented approach; sound decision-making; and a production-minded, hands-on work style focused on customer service and operational excellence.
Experience working closely with production, external partners, and creative development teams.
Production experience with Nutanix, Azure, and AWS, including physical, virtual, and VDI server infrastructure.
Storage and backup expertise with NAS, SAN, LUN, and Rubrik backup platforms.
Expertise in Ansible automation and scripting with Python, PowerShell, and Bash, along with strong Linux and Windows administration skills.
Familiarity with networking concepts including routing, switching, firewalls, and load balancing.
Availability for on-call and overnight support, change management, and patching responsibilities on a rotating team schedule.

Responsibilities

Operate the Azure Virtual Desktop environment, including host pools, session hosts, and user profiles.
Perform AVD provisioning, user onboarding and offboarding, and routine configuration through Nerdio.
Serve as the first point of contact for VDI issues, resolve common performance, access, and profile problems, and escalate other issues.
Apply scheduled updates to AVD golden images according to established procedures.
Monitor AVD capacity and utilization, flag trends, and make routine scaling adjustments.
Monitor server and storage health across bare-metal, Nutanix, and Azure environments; respond to alerts and investigate anomalies.
Perform routine health checks and standard provisioning requests on Qumulo and Infinidat storage platforms.
Monitor Rubrik backup jobs, resolve common failures, and perform restores according to documented procedures.
Support S3 bucket workflows and routine data backup, archiving, and media management tasks.
Perform basic hardware troubleshooting, including component replacement, and assist with vendor support cases and RMAs.
Assist senior engineers with server and storage installations, upgrades, and buildouts.
Provide Level 1 support for infrastructure incidents and service requests, working tickets in ServiceNow and escalating complex or unfamiliar issues with clear notes.
Execute scheduled firmware, software, and operating-system patching, including overnight maintenance windows, according to approved change plans.
Submit change requests for review and carry out approved changes as directed.
Participate in a team-based 24/7 on-call rotation, providing first response and escalating as needed.
Run existing Ansible playbooks and scripts for patching and routine maintenance, and script repetitive tasks in PowerShell with senior engineering support.
Use SolarWinds and other monitoring tools for daily health checks and alert response.
Maintain infrastructure documentation, including diagrams, equipment lists, change records, and current runbooks and procedures, including AVD runbooks; identify missing or unclear procedures.
Work with senior engineering, network, and application teams to troubleshoot cross-system issues.
Provide hands-on infrastructure assistance to studio, production, and live event teams.
Support Genetec CCTV backend server infrastructure and provide coverage across other department areas as needed.
Provide exceptional experiences for guests, partners, and team members, including following on-site appearance and presentation guidelines.