Member of Technical Staff - Research, Cyber

Vals AI • San Francisco, CA

3d • Onsite

About The Position

Vals AI is seeking an exceptional researcher to design and build challenging, high-impact AI benchmarks for cybersecurity. The role focuses on defining how foundation models are evaluated for real-world cybersecurity capabilities and risks. The benchmarks are intended to influence foundation-model development and generative AI applications, and the work involves collaborating with major foundation model labs, financial institutions, and application-layer companies.

The researcher will develop novel benchmarks, ensure they are valid and reliable, analyze model results, and communicate findings to technical and non-technical audiences. The role also includes working with the infrastructure team to implement benchmark designs at scale, publishing research, and keeping current with LLM cybersecurity capabilities and evaluation methods. This is an in-person role in San Francisco; relocation or transportation support is available.

The company describes its founding methodology as originating in NLP evaluation research conducted at Stanford. It reports a $5 million seed round and a recently announced $40 million Series A at a $400 million valuation. Its technology stack includes Python, Django, React, AWS, and CDK for infrastructure as code.

Requirements

• Hands-on cybersecurity experience, such as work as a security engineer, penetration tester, red teamer, or vulnerability researcher.
• Strong grounding in operating systems, cloud infrastructure, and networking.
• Interest in rigorous capability measurement and benchmarking.
• Experience communicating complex research ideas clearly to technical and non-technical audiences.
• Experience working in research teams and integrating feedback.
• Already uses LLMs in security work and wants to work with them regularly.

Nice To Haves

• Experience building cyber evaluations, CTF challenges, or security benchmarks.
• Previous work as a security engineer at a security company or on an in-house security team.
• Experience on an AI or ML team at a security company.
• Strong ties to the security community, such as CTF teams, conference talks, open-source security tools, or research collaborations.
• Published security research, papers, or widely read technical write-ups.

Responsibilities

• Design and develop novel, high-impact benchmarks assessing challenging real-world cybersecurity capabilities.
• Conduct research to ensure benchmarks are valid, reliable, and meaningful.
• Collaborate with foundation model labs and enterprise security teams to understand evolving evaluation needs.
• Analyze model performance across benchmarks and articulate results.
• Publish research findings and contribute to security and evaluation research communities.
• Work with the infrastructure team to implement benchmark designs at scale.
• Stay current with developments in LLM cybersecurity capabilities and evaluation methodologies.

Benefits

• Highly competitive salary and meaningful ownership; the posting states that excellence is well rewarded.
• Relocation and transportation support.
• Health and dental insurance coverage.
• Lunch and dinner provided, plus free snacks, coffee, and drinks.
• Unlimited PTO.
• $1,500 housing stipend for housing within a one-mile radius.
• Opportunity to collaborate with leading AI labs and work on AI development and evaluations.
• Opportunity to publish and present work.

What This Job Offers

• Job Type: Full-time
• Education Level: No Education Listed