Lead Site Reliability Engineer

Morgan Stanley•New York, NY

3d•$155,000 - $215,000

About The Position

The Vice President, Site Reliability Engineering (SRE), leads the stability, resiliency, availability, and operational excellence of critical Wealth Management Technology platforms. The role owns end-to-end production reliability, champions SRE practices, and partners with Engineering, Infrastructure, Operations, and Business stakeholders to support seamless client and advisor experiences. The position leads reliability engineering practices, defines and tracks reliability metrics, and drives improvements to incident detection, identification, and restoration. It provides leadership during production incidents, oversees production governance and change management, and ensures applications are operationally ready and supportable. The role also supports blockchain-connected infrastructure and digital asset systems within a regulated framework. Additional responsibilities include advancing automation and AI-driven operational workflows, maintaining operational documentation and readiness, and advising on architecture, scalability, capacity planning, disaster recovery, and resiliency testing. The position is a technically hands-on leadership role requiring collaboration across technology teams, business stakeholders, vendors, and support teams.

Requirements

10 or more years of experience in a production environment.
Solid software development background and understanding of performance tuning, end-to-end troubleshooting, networking fundamentals, and attention to detail.
BS/MS or equivalent, preferably in a quantitative discipline such as Computer Science or Computer Engineering.
5 or more years of experience leading a small to medium team of similar skillset.
5 or more years of experience driving SRE principles and Chaos Engineering and supporting applications in the Cloud.
2 or more years of experience supporting or deploying applications in AWS/GCP cloud technologies.
2 or more years of experience with Digital Assets/Blockchain-connected infrastructure, digital wallet security systems, or distributed ledger technology pipelines.
Technically hands-on experience understanding both code and infrastructure.
Strong scripting skills, such as Shell scripting, Python, or Perl, and cloud-driven development experience.
Hands-on experience with AI and implementation of AI tools for operational efficiency.
Strong database skills with DB2, Sybase, or Oracle.
Working knowledge of DevOps and observability tools such as Grafana, Prometheus, Splunk, or Kibana.
Strong analytical skills, problem determination, and resolution recovery capabilities.
Ability to build excellent working relationships with technology teams, business analysts, and vendors.
Ability to learn technologies quickly in a fast-paced environment and willingness to learn new technologies and techniques.
Hands-on experience administering large-scale, high-availability systems and tools for monitoring performance and availability.
Excellent communication and writing skills for technical discussions across management layers.
Experience with on-call incidents and ability to respond to emergencies on a 24/7 basis.
Strong ownership mentality focused on customer satisfaction.
Ability to manage outage incidents, coordinate user communications, and work with other teams to resolve incidents.

Nice To Haves

Experience working within the Financial Services Industry.

Responsibilities

Champion SRE principles, including Service Level Indicators (SLIs), Service Level Objectives (SLOs), Error Budgets, and operational risk management frameworks.
Define and track reliability metrics measuring platform health, customer experience, and operational effectiveness.
Lead initiatives to reduce Mean Time to Detect (MTTD), Mean Time to Identify (MTTI), and Mean Time to Restore (MTTR).
Lead proactive detection, triage, and resolution of production issues affecting business-critical applications and services.
Own escalated production incidents and coordinate resolution across application, infrastructure, vendor, and external partner teams until service is restored and client impact is mitigated.
Promote operational excellence focused on stability, resiliency, and continuous service improvement.
Communicate clearly and promptly during outages, including business impact assessments and recovery updates to senior leadership.
Support blockchain-connected infrastructure, node operations, digital wallet security systems, or distributed ledger technology pipelines within a regulated framework.
Ensure production changes comply with change management policies, release controls, operational readiness standards, and risk management practices.
Assess the operational impact of technology changes and ensure appropriate testing, rollback strategies, monitoring, and support models are in place before production deployment.
Partner with development teams throughout the software lifecycle to build reliability, observability, and operational supportability into applications and services.
Identify and lead opportunities to reduce manual effort and operational toil through automation, self-healing capabilities, and AI-driven operational workflows.
Promote engineering-led operations and continuous process optimization.
Establish and maintain accurate, current, accessible runbooks, troubleshooting guides, standards, and operational procedures.
Drive operational readiness programs that improve first-level diagnosis and reduce reliance on development teams for routine issue resolution.
Create end-to-end Know Your System diagrams.
Ensure support teams maintain high-quality documentation and standardized troubleshooting practices.
Advise on reliability, resiliency, observability, and production support strategies.
Guide architecture reviews, platform scalability, capacity planning, disaster recovery, and resiliency testing.
Partner with engineering teams to identify systemic risks and implement long-term improvements to platform stability and customer experience.
Lead a small to medium team of similar skillset.
Respond to on-call incidents and emergencies on a 24/7 basis, coordinating user communications and teams during outages.