Principal AI SOC Engineer

Quantum Sky•Reston, VA

3d•$200,000 - $225,000•Hybrid

About The Position

Quantum Sky is searching for a Principal AI SOC Engineer to drive efficiency, velocity, and operational effectiveness across Security Operations Centers supporting federal missions. This is a hands-on engineering role for someone who has worked inside a federal watch floor, has built and integrated SOC tooling at enterprise scale, and knows how to turn analyst pain points into production-grade automated capability. The ideal candidate is a builder and a leader: an engineer who writes production Python daily, is deeply fluent in Splunk, and has carried AI/ML and GenAI capabilities through a real software development lifecycle — versioned, tested, evaluated, monitored, and governed — inside a federal agency's accreditation and oversight regime. The role blends technical execution with engineering leadership, including backlog ownership in Jira, design documentation in Confluence, delivery oversight, code review, and mentoring.

Requirements

8–12 years of hands-on enterprise IT and cybersecurity engineering experience spanning security operations, cloud platforms, automation, and AI/ML.
3–5 years of direct SOC engineering experience designing, building, and optimizing SOC tooling, with at least 2 years supporting federal SOC environments (civilian agencies, DoD, Intelligence Community, or federal law enforcement).
Expert-level Python: production services, automation frameworks, API integrations, unit and integration testing, packaging, and code review of others' work.
Deep hands-on Splunk expertise: advanced SPL, data model and CIM work, Splunk Enterprise Security content development, ingest pipeline and architecture design, and Splunk SOAR playbook development.
3+ years applying AI/ML techniques to cybersecurity or operational systems, including AI-enabled workflows for alert enrichment, triage, detection engineering, or automated response.
2+ years working with LLMs or GenAI systems in production or near-production environments, including RAG pipelines and LLM-integrated automation for SOC use cases.
Demonstrated AI SDLC ownership: taking an AI/ML or GenAI capability from requirement through evaluation, deployment, and sustained operation, with versioning, reproducibility, testing, and monitoring in place.
Strong experience with SOC automation, orchestration, and playbook design, including API-driven integrations and detection, correlation, and response pipeline engineering.
Proficiency with Git-based workflows, CI/CD pipelines, and containerized deployment.
Working fluency with Jira and Confluence as engineering delivery and documentation tools — backlog ownership, sprint execution, and written design and decision records.
Working knowledge of the federal cybersecurity compliance stack: FISMA, NIST 800-53, RMF and the ATO process, FedRAMP for cloud services, and CISA directive-driven operations.
Demonstrated ability to set technical strategy, review designs, and foster best practices across a team of engineers.
Must have a current TS/SCI

Nice To Haves

Prior engineering support to a large, federated federal civilian agency SOC or a federal law enforcement environment.
Familiarity with current OMB AI governance requirements, NIST AI RMF, and federal AI use case inventory and impact-assessment processes.
Experience with TIC 3.0, CDM, and CISA incident reporting workflows.
Experience deploying AI/ML workloads in FedRAMP-authorized environments, and experience self-hosting open-weight models where data sensitivity precludes commercial API use.
Familiarity with CJIS Security Policy and handling of law enforcement sensitive data.
Infrastructure-as-code and configuration management experience (Terraform, Ansible, Kubernetes).
Experience with Azure and Microsoft Sentinel, including Azure OpenAI or Azure ML for GenAI use cases.
Certifications: Splunk Enterprise Certified Architect, Splunk SOAR Certified Automation Developer; CISSP; GCIA, GCDA, or GCFA; Microsoft Certified: Azure Security Engineer Associate.

Responsibilities

Design and engineer AI-powered SOC capabilities that improve analyst efficiency, reduce alert fatigue, and accelerate detection and response.
Apply AI/ML and GenAI techniques to concrete SOC problems: alert enrichment, triage and prioritization, entity and campaign correlation, investigation summarization, phishing and insider-threat triage, and automated response recommendation.
Own the full AI SDLC for delivered capabilities — problem framing, data curation and labeling, model or prompt development, evaluation harness design, CI/CD integration, ATO-compatible deployment, monitoring for drift and regression, and rollback.
Build evaluation and test methodology for AI-enabled workflows: golden datasets, regression suites, precision/recall and false-positive measurement, and human-in-the-loop review gates before any capability influences analyst action or containment.
Implement guardrails, output validation, prompt and response logging, and decision traceability so AI-assisted findings are auditable and defensible to agency leadership, oversight bodies, and assessors.
Engineer AI capabilities consistent with federal AI governance expectations — current OMB AI guidance, agency Chief AI Officer requirements, AI use case inventory reporting, and NIST AI RMF — including the additional practices that apply when a capability is designated high-impact.
Ensure AI capabilities handle sensitive government data appropriately, including PII and law enforcement sensitive material, with data minimization, retention controls, and model-training exclusions.
Build and evolve detection and response pipelines across SIEM, SOAR, EDR, email security, identity, and cloud security platforms, with Splunk as the primary analytic platform.
Engineer Splunk content and infrastructure: advanced SPL, data models and CIM normalization, correlation searches and notable event tuning in Enterprise Security, ingest and index architecture, and performance tuning for high-volume telemetry.
Build and maintain SOC automation in Python — SOAR playbooks, custom microservices, API-driven integrations, and AI-driven decisioning across security and infrastructure controls.
Engineer onboarding and normalization pipelines for telemetry from organizationally distinct components with heterogeneous tooling, ownership models, and data-sharing constraints.
Support enterprise event logging maturity requirements, including log source coverage, retention tiering, and log integrity.
Build and maintain integrations supporting federal reporting and directive compliance, including CISA sensor and CDM data flows, Binding Operational and Emergency Directive response, and incident notification timelines.
Improve SOC velocity and throughput by automating repetitive analyst tasks and standardizing response patterns into reusable, tested components.
Support design of scalable SOC architectures for high-volume telemetry and real-time workflows in a 24/7 operations environment.
Maintain a tool-agnostic engineering mindset; integrate Elastic, Microsoft Sentinel and Defender, or cloud-native services where the mission calls for it.
Own and groom the engineering backlog in Jira; prioritize, decompose, estimate, and ship production-ready increments on a predictable cadence against contract deliverables.
Translate analyst user stories and operational requirements into concrete technical designs; document architecture decisions, runbooks, SOPs, and capability documentation in Confluence.
Set engineering standards and patterns for AI-enabled SOC capabilities — code review expectations, testing requirements, repository structure, CI/CD pipelines, and secure development practices aligned to NIST SSDF.
Partner with government stakeholders, analysts, and fellow engineers to deliver solutions, setting a high technical bar through hands-on contribution and shared ownership.
Mentor engineers on Python, Splunk, and AI engineering practice.