Splunk Systems Engineer - Hybrid

MSP • Richmond, VA

4d • Hybrid

About The Position

Genesis10 is seeking a Splunk Systems Engineer for a hybrid role with a Global Financial Institution. The engineer will act as a trusted platform owner, ensuring Splunk availability, scalability, and reliability while partnering closely with Information Security, SOC, architecture, engineering, and operations teams. This role will own end-to-end production support for a highly distributed Splunk Enterprise and Splunk Cloud environment.

Requirements

• 5+ years of hands-on experience administering large-scale Splunk Enterprise or Splunk Cloud environments
• Strong expertise in: Indexer clustering and search head clustering, Universal and heavy forwarder architectures, SmartStore / S3-compatible object storage, SPL, search optimization, summary indexing, and data model acceleration
• Deep experience with security log ingestion and SIEM use cases
• Proven ability to lead production incidents, perform RCA, and drive preventive solutions
• Strong Linux administration skills and experience managing Splunk configuration and apps
• Experience working in 24x7 production environments with high availability expectations
• Excellent written and verbal communication skills, with the ability to engage senior technical and business stakeholders

Nice To Haves

• A production owner's mindset with deep technical credibility in Splunk and data pipelines
• Splunk certifications such as Enterprise Admin or Enterprise Architect
• Experience with Splunk Enterprise Security (ES) and SOAR (Phantom or equivalent)
• Exposure to cloud logging and security architectures (AWS, Azure, GCP)
• Knowledge of Red Hat Enterprise Linux and Windows Server administration
• Experience with monitoring, APM, and event management tools
• Strong understanding of security, network, system, and database operations

Responsibilities

• Own end-to-end production support for a highly distributed Splunk Enterprise and Splunk Cloud environment, including search head clusters, indexer clusters, deployers, deployment servers, and forwarders
• Ensure high availability, performance, and resiliency of the Splunk platform supporting security and operational use cases
• Lead incident response, troubleshooting, root cause analysis (RCA), and service restoration for Splunk and Cribl platforms
• Proactively identify risks, capacity constraints, and performance bottlenecks; implement preventive and tuning measures
• Serve as a key technical enabler for Information Security and SOC teams, ensuring timely, accurate, and reliable ingestion of security logs
• Onboard and normalize new data sources, supporting CIM compliance, field normalization, and SIEM best practices
• Tune ingestion pipelines using props.conf and transforms.conf, index-time and search-time optimizations
• Build and support dashboards, searches, and alerts that enable threat detection, investigations, and reporting
• Administer and support the Cribl environment for data routing, filtering, enrichment, and cost optimization
• Develop and maintain runbooks, SOPs, installation guides, and operational documentation
• Adhere to change management, incident management, and SLA commitments using ITSM tools

Benefits

• Access to hundreds of clients, most who have been working with Genesis10 for 5-20+ years.
• The opportunity to have a career-home in Genesis10; many of our consultants have been working exclusively with Genesis10 for years.
• Access to an experienced, caring recruiting team (more than 7 years of experience, on average.)
• Behavioral Health Platform
• Medical, Dental, Vision
• Health Savings Account
• Voluntary Hospital Indemnity (Critical Illness & Accident)
• Voluntary Term Life Insurance
• 401K
• Sick Pay (for applicable states/municipalities)
• Commuter Benefits (Dallas, NYC, SF, and Illinois)

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed