About The Position
We're looking for cybersecurity experts to help train and evaluate the next generation of AI systems. In this role, you'll apply your technical expertise to create realistic cybersecurity scenarios, assess AI-generated responses, and help ensure models are accurate, useful, and responsible when handling complex security topics. You'll work alongside a multidisciplinary team improving how AI understands software security, cyber defense, vulnerability research, and related technical domains. Your expertise will directly shape the quality, safety, and reliability of AI systems used by millions.
Requirements
- Bachelor's or Master's degree in Computer Science, Cybersecurity, or a closely related technical field OR 5+ years of professional software engineering experience at a high-growth startup or established technology company.
- Deep knowledge of cybersecurity principles, modern software systems, and secure development practices.
- Excellent technical reasoning and strong written communication skills in English.
- Sound judgment when working with sensitive or dual-use security topics.
- The ability to apply detailed guidelines while maintaining high-quality, consistent evaluations.
Nice To Haves
- Experience in offensive security, penetration testing, vulnerability research, security engineering, or a related discipline.
- Previous experience evaluating, reviewing, benchmarking, or red-teaming technical or AI-generated content.
- Familiarity with AI evaluation, model assessment, or human-in-the-loop data annotation workflows.
Responsibilities
- Design realistic, expert-level cybersecurity prompts spanning areas such as application security, network security, cloud infrastructure, vulnerability management, and secure software development.
- Evaluate AI-generated responses for technical accuracy, completeness, and practical usefulness.
- Assess how models handle sensitive and dual-use cybersecurity scenarios, ensuring responses are responsible and aligned with project guidelines.
- Apply structured evaluation rubrics to annotate prompts and model outputs with consistency and precision.
- Provide clear, actionable feedback that helps improve the performance, safety, and reliability of frontier AI systems.
Benefits
- The hourly rate will depend on experience and project parameters. The rate will be disclosed prior to starting the project.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
Upload and Match Resume
What This Job Offers
Career Level
Senior
