Application Security Architect - AI Harness

Edward Jones

9d•Hybrid

About The Position

The Application Security Architect, Agentic Secure Code Architect is a hands-on architecture-focused IC responsible for designing, integrating, maintaining, and improving a CI/CD-integrated AI evaluation harness used to assess application and infrastructure source code for security vulnerabilities and insecure-design practices. This role combines AppSec, DevSecOps, AI engineering, secure SDLC governance, and financial-services compliance, with strong emphasis on source-code protection, enterprise controls governance, repeatability, auditability, measurable risk reduction, and developer trust.

Requirements

Bachelor’s degree in Computer Science, Cybersecurity, Software Engineering, Information Technology, Engineering, or related field.
10+ years of experience in application security, secure software engineering, DevSecOps, security architecture, or related cybersecurity roles.
Deep expertise in secure code review, vulnerability detection, threat modeling, exploitability analysis, and secure SDLC practices.
Experience with performing vulnerability and penetration test readouts/walkthroughs with stakeholders.
Hands-on experience with SAST, SCA, DAST, secrets scanning, API security testing, container security, infrastructure-as-code scanning, and developer workflow integrations.
Experience integrating security capabilities into SDLC pipeline tooling: Jenkins, GitHub Actions, GitHub Enterprise, Atlassian, developer portals, source-control workflows, and DevSecOps toolchains.
Practical experience using LLMs or AI models for code review, software engineering, vulnerability research, security analysis, or developer productivity use cases.
Understanding of prompt engineering and optimization, RAG, model evaluation, AI guardrails, human-in-the-loop review, prompt/model versioning, and vendor/open-source model trade-offs.
Experience designing or maintaining evaluation harnesses, benchmark suites, regression tests, validation pipelines, and test orchestration workflows.
Strong understanding of concepts: OWASP, CWE, CVSS, NIST SSDF, AI security risks, regulated source-code handling, auditability, and vendor/model governance.

Nice To Haves

Master’s degree or relevant certifications such as CISSP, CSSLP, OSWE, GWAPT, CCSP, cloud security, Kubernetes security, or AI governance certifications.
Experience delivering AI-assisted AppSec capabilities in a Fortune 500 or highly regulated financial services environment.
Hands-on experience with enterprise AI platforms, approved vendor models, open-source models, secure hosting, RAG, and AI governance controls.
Experience building secure code review automation, custom static analysis rules, vulnerability detection pipelines, benchmark datasets, or AI-based developer tooling.
Demonstrated success reducing AppSec risk through improved detection accuracy, lower false positives, faster remediation, and stronger developer enablement.

Responsibilities

Architect the CI/CD-integrated AI secure-code evaluation harness as a hands-on IC for source code repositories and Secure SDLC lifecycles.
Integrate evaluation workflows with pipeline tooling and AppSec reporting platforms.
Design AI-assisted secure code review methods that complement SAST, SCA, DAST, secrets scanning, IaC scanning, threat modeling, security testing, and manual assessments.
Maintain benchmarks, golden test cases, prompt/model versions, retrieval configurations, scoring criteria, and regression tests for AI-generated findings.
Govern approved vendor and frontier AI models, including selection, routing, fallback patterns, accuracy, explainability, cost, and data-protection trade-offs.
Define safeguards for proprietary production code, including access controls, approved model endpoints, minimization, retention limits, secure logging, and evidence handling.
Route validated findings into developer workflows with actionable remediation guidance and feedback loops to reduce false positives and improve adoption.
Define metrics and control evidence aligned to internal governance processes, financial services industries authorities, and cyber security frameworks (NIST SSDF, NIST CSF 2.0, NYDFS, FINRA, SOX ITGC, FFIEC, GLBA).