SRE Splunk Enterprise Administrator – (w/ active Secret)

Critical Solutions•Norfolk, VA

3d•$68,000 - $87,000•Onsite

About The Position

Critical Solutions is seeking an SRE Splunk Enterprise Administrator to support a federal customer. The administrator will join the Site Reliability Engineering team supporting mission-critical cybersecurity operations by administering and maintaining distributed Splunk Enterprise platforms across hybrid-cloud and on-premises environments. The role covers daily platform operations, performance and ingestion monitoring, data-onboarding support, incident troubleshooting, security compliance, documentation, and modernization in an Agile/DevOps operating culture. The position is 100% onsite in Norfolk, Virginia, at Norfolk Station, Monday through Friday on the day shift. The employee must participate in after-hours support and an on-call rotation as required. Candidates must be U.S. citizens with an active DoD Secret clearance or higher and must complete the federal agency’s background investigation and fingerprinting. The role requires a bachelor’s degree with relevant experience, or a master’s degree with relevant experience, at least three years administering Splunk Enterprise v9 in distributed production environments, DoD 8570.01 IAT Level II certification, and Splunk Enterprise Certified Admin or Splunk Cloud Certified Admin certification. The successful candidate will support platform operations, compliance, modernization, and reliability improvements, including work measured by service-level objectives, reduced recovery time, testing coverage, performance, and scalability.

Requirements

Must be a U.S. citizen.
Must possess an active DoD Secret Security Clearance or higher.
Must undergo and successfully pass the federal agency background investigation and fingerprinting process.
Bachelor’s degree with prior relevant experience, or a master’s degree with prior relevant experience.
Minimum DoD 8570.01 IAT Level II certification.
Splunk Enterprise Certified Admin or Splunk Cloud Certified Admin certification.
At least three years of experience administering Splunk Enterprise v9 and supporting distributed Splunk architectures in hybrid-cloud and on-premises production environments.
Experience administering and troubleshooting Red Hat Enterprise Linux (RHEL) 8 and/or RHEL 9 servers.
Experience with STIGs, the Risk Management Framework (RMF), vulnerability management, and compliance frameworks.
Experience developing and maintaining operational dashboards, visualizations, and executive reporting in Splunk Enterprise and Splunk Cloud.
Strong working knowledge of Splunk data-ingestion pipelines, indexing, parsing, forwarding, search optimization, and retention management.
Experience using at least one scripting or automation language: Python, Bash, or PowerShell.
Strong troubleshooting, analytical, communication, documentation, and problem-solving skills.
Ability to work independently, manage competing priorities, collaborate across technical teams, and maintain a customer-focused approach in a high-tempo production environment.
Ability to work onsite at Norfolk Station Monday through Friday on the day shift.
Ability to participate in after-hours support and an on-call rotation as required.

Nice To Haves

Experience with automated script design, coding, debugging, and maintenance using Bash, Python, or similar tools.
Experience working in classified government or Department of Defense environments.
Familiarity with RHEL 10.

Responsibilities

Administer, maintain, and perform daily operations and maintenance for distributed Splunk Enterprise environments, including Search Heads, Indexers, Heavy Forwarders, Intermediate Forwarders, Universal Forwarders, and Deployment Servers across hybrid-cloud and on-premises infrastructure.
Monitor platform health, data ingestion, indexing throughput, search performance, and retention utilization; troubleshoot data-onboarding, parsing, field-extraction, forwarding, indexing, and ingestion issues.
Support Splunk Cloud integrations and related hybrid operational activities.
Maintain Splunk applications, dashboards, alerts, saved searches, and knowledge objects.
Support patching, vulnerability remediation, STIG compliance, and system-hardening activities.
Support platform upgrades, migrations, infrastructure modernization, technology refresh, and automation initiatives using scripting and configuration-management tools.
Participate in incident response, outage troubleshooting, problem resolution, and root-cause analysis.
Maintain operational documentation, architecture diagrams, runbooks, and standard operating procedures.
Coordinate with cybersecurity, network, server, engineering, and customer teams during operational and modernization activities.
Participate in after-hours support and an on-call rotation as required.
Contribute to improved system reliability, updated automated test coverage, timely resolution of performance bottlenecks, CI/CD testing integration, and improved scalability and performance under high load.