Member of Technical Staff - Research, Cyber

Vals AI • San Francisco, CA

11h • $130,000 - $185,000 • Onsite

About The Position

Vals AI is seeking an exceptional researcher to design and build next-generation AI benchmarks for cybersecurity. The role focuses on developing challenging evaluations that measure real-world capabilities and risks of large language models (LLMs), helping shape how foundation models are developed and generative AI applications are built. The researcher will help define standards for evaluating LLMs on real-world tasks and work with foundation model labs, financial institutions, and application-layer companies.

The position is based on an in-person team in San Francisco. Relocation or transportation support is available. The role also involves working with the infrastructure team to implement benchmark designs at scale, analyzing and communicating results, and publishing research findings.

Requirements

• Hands-on cybersecurity experience, such as work as a security engineer, penetration tester, red teamer, or vulnerability researcher; experience finding and reporting vulnerabilities through bug bounty programs or CVEs is relevant.
• Strong grounding in operating systems, cloud infrastructure, and networking.
• Interest in rigorous capability measurement; prior work building cyber evaluations, CTF challenges, or security benchmarks is described as ideal.
• Already uses LLMs in security work and wants to work with them every day.
• Ability to communicate complex research ideas clearly to technical and non-technical audiences.
• Experience working in research teams and integrating feedback.
• Ability to learn new skills and technologies quickly.
• Initiative and ownership in a small team, with an autonomous and solution-oriented approach.
• Capacity to execute at high speed in a fast-changing LLM landscape.

Nice To Haves

• Previous work as a security engineer at a security company or on an in-house security team, and/or experience on an AI or ML team at a security company.
• Strong ties to the security community, such as CTF teams, conference talks, open-source security tools, research collaborations, published security research, papers, or widely read technical write-ups.

Responsibilities

• Design and develop novel, high-impact benchmarks assessing challenging real-world cybersecurity capabilities.
• Conduct research to ensure benchmarks are valid, reliable, and meaningful.
• Collaborate with foundation model labs and enterprise security teams to understand evolving evaluation needs.
• Analyze model performance across benchmarks and articulate the results.
• Publish research findings and contribute to security and evaluation research communities.
• Work with the infrastructure team to implement benchmark designs at scale.
• Stay current with developments in LLM cyber capabilities and evaluation methodologies.

Benefits

• Highly competitive salary and meaningful ownership.
• Relocation and transportation support.
• Health and dental insurance coverage.
• Lunch and dinner provided, with free snacks, coffee, and drinks.
• Unlimited PTO.
• $1,500 housing stipend for living within a one-mile radius.
• Opportunity to collaborate with leading AI labs and work on the front lines of AI development and evaluations.
• Opportunity to publish and present work.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed