Senior Splunk ITSI Engineer

Olympus Solutions Inc. • Washington, DC

2d • $165,000 - $195,000 • Onsite

About The Position

Olympus Solutions, Inc. is seeking a highly motivated Senior Splunk ITSI Engineer to own the service intelligence layer of a federal customer’s Splunk environment. In this hands-on engineering role, the engineer will model mission and IT services in Splunk IT Service Intelligence (ITSI), design KPIs and thresholds that inform service health scores, tune event analytics to reduce noise, and maintain the performance of a search-heavy ITSI deployment. The engineer will serve as the customer’s primary ITSI subject matter expert and work onsite with government and contractor operations staff.

The position is 100% onsite at a government facility in the Maryland/Washington, DC metro area; remote and hybrid work are not available. The schedule is standard business hours, with occasional after-hours support for upgrades and incident response. Travel is minimal. This is a full-time, exempt position with an annual salary range of $165,000 to $195,000, depending on ITSI experience, Splunk certifications, and overall qualifications. An active TS/SCI clearance is required at application, and U.S. citizenship is required. Olympus will not sponsor an initial clearance.

Olympus Solutions is a Security Systems Engineering company serving federal, defense, commercial, and state, local, and education clients. The company specializes in identity systems and related security engineering, implementation, operations, maintenance, and sustainment. It is a Service-Disabled Veteran-Owned Small Business and an Equal Opportunity Employer.

Requirements

• Active TS/SCI security clearance at the time of application; Olympus will not sponsor an initial clearance.
• U.S. citizenship.
• At least 8 years of IT operations, systems engineering, or monitoring engineering experience, including at least 5 years of hands-on Splunk engineering.
• At least 2 years building and operating Splunk ITSI in production, including service modeling, KPI design, and event analytics.
• Strong Search Processing Language (SPL) skills, including base searches, data model acceleration, and search optimization.
• Linux administration and scripting skills in Python and Bash in support of Splunk.
• Experience integrating Splunk with an ITSM platform.
• Bachelor’s degree in Computer Science, Information Systems, or a related field, or four additional years of relevant experience in lieu of a degree.
• Ability to work 100% onsite at a government facility in the Maryland/Washington, DC metro area.

Nice To Haves

• Splunk ITSI Certified Admin.
• Splunk Enterprise Certified Admin or Splunk Enterprise Certified Architect.
• Experience with Splunk Enterprise Security or Splunk Observability Cloud and how each interoperates with ITSI.
• Experience supporting DoD, Intelligence Community, or federal civilian operations centers.
• Working knowledge of ITIL service management and AIOps concepts.
• Experience running Splunk on AWS GovCloud or Azure Government.
• DoD 8140 baseline certification (Security+ or higher).
• Experience integrating Splunk with ServiceNow, which is preferred.

Responsibilities

• Design, build, and maintain ITSI service trees, including service decomposition, entity definition and discovery, dependency mapping, and service templates.
• Develop KPIs and shared KPI base searches; configure static and adaptive (machine learning) thresholds, time policies, and health score weighting.
• Create and maintain glass tables, deep dives, and service analyzer views for NOC, operations, and executive audiences.
• Build and tune event analytics, including correlation searches, notable event aggregation policies, episode review workflows, and ITSM integration for ticketing.
• Onboard and normalize data sources feeding ITSI, including CIM compliance, entity enrichment, and content pack deployment.
• Monitor and tune ITSI performance, including search scheduler load, KPI search concurrency, backfill, and summary index health.
• Administer the underlying Splunk Enterprise or Splunk Cloud platform as needed, including search head and indexer configuration, apps and add-ons, role-based access control, and version upgrades.
• Support the environment’s security posture, including STIG application, audit logging, and change management.
• Document service models, KPI definitions, runbooks, and standard operating procedures.
• Brief government stakeholders on service health and monitoring maturity.
• Mentor junior Splunk staff and provide technical direction on monitoring strategy.
• Provide occasional after-hours support for upgrades and incident response.

Benefits

• Comprehensive medical, dental, and vision benefits for employees and their families.
• Paid time off plan and a culture that values balance, accountability, and personal well-being.
• Career-building opportunities, professional development, and support for training and certifications.
• 401(k) plan with a discretionary company match and no waiting period before enrollment.
• Eligibility for referral bonuses for qualified new-hire and business development referrals.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior