Operations Engineering, Senior Advisor

Peraton•Baltimore, MD

1d•$176,000 - $282,000

About The Position

The Production Operations Lead is accountable for the stability, availability, and recoverability of MARx in production. MARx is the CMS system of record for Medicare Advantage and Part D enrollment, premiums, and plan payments, serving about 53 million beneficiaries and more than 300 Plan Sponsor Organizations and supporting about $68 billion in monthly plan payments. The Lead runs day-to-day production operations across the CMS AWS environment; leads incident and problem management; owns disaster recovery and continuity; and ensures the operational calendar—including daily, monthly, and annual processing, payment files, file exchanges, and New Year Preparation—is met on time. The role also drives automation and process improvements to shift team capacity from operations and maintenance toward enhancements. It works closely with CMS business owners, CMS OIT, MBOSC, CMS help desks and incident response teams, and partner systems including SSA, RRB, HPMS, APPS, and the Common Medicare Environment shared with MEPBS. The Lead makes operational prioritization decisions under pressure, balancing ongoing ticket triage, security incidents, New Year Preparation, and enhancement work while protecting payment accuracy, beneficiary coverage, and on-time delivery to plans. Beyond production operations, the Lead serves as a trusted operational advisor to program leadership and CMS, mentors production support staff, and leads reliability and automation improvements. The position is contingent on contract award.

Requirements

At least 12 years of professional experience with a BS/BA, 10 years with an MS/MA, or 7 years with a Ph.D.
Experience leading production operations for a large, mission-critical AWS system, including monitoring, incident response, and high-availability operations.
Demonstrated experience leading production support teams and coordinating incident response across multiple technical teams and external partners.
Experience owning disaster recovery and continuity, including DR playbooks, RTO/RPO, contingency-plan testing, and failover/failback execution.
Experience with release and change management in a SAFe/Agile environment with CI/CD pipelines.
Experience supporting federal government systems that process personally identifiable information and financial transactions.
Ability to obtain and maintain the CMS background investigation and HSPD-12 PIV credential required for the position.
Experience operating batch processing and file-based data exchanges with external partners on fixed processing calendars.
Working knowledge of federal security operations, including FISMA, NIST SP 800-53, incident reporting, vulnerability remediation, POA&M management, and ATO support.
Hands-on knowledge of AWS operations, including compute, containers, serverless, databases, networking, monitoring, and security services.
Experience with observability tooling, centralized logging, alerting, and on-call management.
Experience with production database operations, including availability, backup, recovery, and data integrity.
Understanding of CMS and Medicare program operations and how production issues affect beneficiaries, plans, and payments.

Nice To Haves

Experience with CMS systems and processes, especially Medicare Advantage and Part D enrollment, premium, and payment operations, and the New Year Preparation cycle.
Hands-on experience with MARx operations tools, including Splunk, Splunk On-Call, Datadog, CloudWatch, AWS Security Hub and GuardDuty, and Amazon QuickSight.
Familiarity with Aurora MySQL and PostgreSQL, ECS Fargate, Lambda, AWS Batch, SQS/SNS, and CMS Enterprise File Transfer.
Industry certification such as AWS Certified SysOps Administrator or DevOps Engineer, ITIL 4, or equivalent.
Experience leading SRE-style automation that measurably reduced operational workload.
Direct experience with MARx production operations, including its batch schedule, interfaces, and operational calendar.
Familiarity with CMS governance, including the Target Life Cycle, Technical Reference Architecture, Acceptable Risk Safeguards, and MEPS Change Control Board process.
Experience with HHS/CMS incident and breach response procedures, including breach reporting under OMB M-17-12.
Experience with ServiceNow incident and case workflows.
Experience applying SI-enabled monitoring or automation, such as anomaly detection, in a regulated environment.

Responsibilities

Lead real-time monitoring of MARx jobs, system health, cost and usage, and security events using CMS-approved tools; identify and resolve anomalies before they affect plans or beneficiaries.
Maintain and report on the target of 99.9% production system availability and response time.
Maintain scalability and code portability in the cloud environment in collaboration with the DevOps Lead and Architectural Lead, and apply the AWS Well-Architected Framework to production operations.
Plan capacity for peak processing periods, including monthly payment processing, the Annual Enrollment Period, and New Year Preparation; identify processing-error causes and resolve them quickly.
Own and maintain the MARx Disaster Recovery Playbook, including roles, escalation, testing, notifications, infrastructure and network details, and failover and failback procedures.
Meet Business Impact Assessment Recovery Time Objectives and Recovery Point Objectives; lead annual contingency-plan tabletop tests and CMS-requested DR exercises, and track corrective actions.
Lead production incident response from detection through recovery, coordinating scrum teams, DevOps and Security Leads, and partner systems.
Ensure suspected or confirmed security and privacy incidents and breaches are reported to the CMS Incident Response Team, COR, and CO within one hour; respond to HHS CSIRC and CMS IRT alerts and indicators of compromise within 24 hours.
Support incident and breach response by supplying requested logs, images, and event data, and producing post-incident reports covering impact, threat vector, information affected, lessons learned, and mitigation.
Lead root cause analyses for production defects and track preventive actions through closure; communicate releases, maintenance, outages, and downtime, and maintain the downtime communications playbook.
Direct daily, monthly, quarterly, and annual operations, including monthly plan payment file transmissions to APPS and inbound and outbound data exchanges with MAOs, SSA, RRB, and internal partners.
Oversee shared Split and Push file transmission, File Mover processes, and partner exchanges defined in the Plan Communications User Guide; identify exchange issues and test and implement fixes.
Lead operational readiness for New Year Preparation, the Beneficiary Rate Increase period from July through March, and the Annual Enrollment Period.
Coordinate semiannual lower-environment data refreshes, de-identification hand-offs, and semiannual and qualifying-beneficiary data archival in accordance with CMS retention requirements.
Oversee premium withhold and premium reduction requests with SSA and RRB across daily, weekly, bi-monthly, monthly, and annual schedules.
Govern production deployments under the twice-weekly release cadence, including production readiness reviews, post-implementation reviews, and rollback readiness; ensure changes follow the CMS Target Life Cycle and MEPS Change Control Board process.
Serve as the operations point of contact for MBOSC, provide SME support on urgent tickets, lead working sessions, deliver time-sensitive data fixes, and keep ticket status visible across MARx, MBOSC, and CMS.
Maintain help desk scripts with the MAPD Help Desk, CMS IT Service Desk, and other CMS help desks; triage aging tickets during transition-in and develop a backlog reduction plan.
Drive the aging ticket backlog toward the CMS goal of having no open tickets more than one year old by the beginning of 2029.
Ensure production vulnerabilities and findings are remediated within HHS POA&M timelines, and track items that cannot meet those timelines in the POA&M.
Support maintenance of the MARx Authority to Operate and CMS Acceptable Risk Safeguards controls, penetration testing, vulnerability scans, and mandated audits, including CFO and A-123 audits.
Manage resource usage to limit CMS costs using cost and usage dashboards; identify and implement automation, including responsible use of Super Intelligence under the approved compliance and risk plan.
Participate in CMS innovation efforts, recommend operational improvements, and evaluate appropriate SI and machine-learning applications for production operations.
Maintain working knowledge of MARx business functions and the operational consequences of failures affecting enrollment, premiums, withholding, payments, reports, data files, and New Year Preparation.
Oversee production database availability, backup, and recovery for MySQL Community, Aurora MySQL, and Aurora PostgreSQL in line with Recovery Point Objectives; promote data integrity and prevent corruption and data-cleanup incidents.
Work with CMS business owners and OIT on operational priorities, production issues, and changes; coordinate with partner and upstream system teams on exchange schedules, incidents, and fixes.
Explain operational risks, incident status, and trade-offs to technical and nontechnical audiences, and facilitate operational communication across CMS groups, states, and federal agencies.
Participate in MEPS Program Increment Planning and follow the MEPS iteration schedule; plan operations capacity alongside enhancement work and advise the Program Manager on unplanned-work priorities.
Document deviations from the PI plan and report them to CMS OIT staff; provide input to risk and issue management and quarterly QASP self-assessments.
Maintain current operations documentation, including SOPs, data fix documents, data validation and verification documents, production readiness reviews, and post-implementation reviews.
Mentor production support analysts and engineers, distribute operational knowledge across scrum teams, contribute to MARx training materials, and train CMS staff as needed.