AI Studio Coach- LLM Red Teaming

Break Through Tech•New York, NY

1d•Remote

About The Position

Break Through Tech transforms how tech talent is discovered, developed, and connected to opportunity in the age of AI. Working across higher education and industry, Break Through Tech delivers programs nationwide and integrates technical training, work-based learning, mentorship, and career readiness with deep employer partnerships to create more effective and scalable routes into early tech careers. Break Through Tech is actively shaping the field of early tech talent development by identifying what works, translating those insights into replicable models, and helping define more consistent approaches to how talent is prepared, assessed, and hired across institutions and employers. At a moment when AI is reshaping every aspect of work and society, Break Through Tech’s mission is urgent: to ensure that the people who build what comes next — and who are in the room where the most consequential technology decisions of our time get made — look like the world they will shape. Learn more about our work at www.breakthroughtech.org/. We are seeking subject matter experts to partner with Break Through Tech program staff to deliver a first-of-its-kind program for up to 500 undergraduate fellows enrolled in our flagship, extracurricular AI Program. As LLM capabilities increase and societal and business risks become more apparent, foundation labs and enterprises are investing heavily in AI safety and security, creating fast-growing demand for ethical “red team” talent. Break Through Tech’s new 16-week LLM Red Teaming program will enable our fellows – who will have already completed a rigorous 9-week Machine Learning Foundations course and a 3-month AI project sponsored by an industry partner – to build the knowledge, mindset, and skills to ethically probe large language models for safety and security vulnerabilities. With expert guidance and support from their Coaches, and within sanctioned platforms, fellows will red team frontier AI models using innovative jailbreaking techniques across a variety of attack surfaces; build a personal portfolio of documented breaks; and compete in a red teaming competition. Across 4 units – Red Teaming Foundations, Direct Prompt Injection, Indirect Prompt Injection, and Live Competition Prep – fellows will learn and apply the techniques that professional and amateur red teamers use in an ethical white hat capacity. Asynchronous coursework will be complemented by biweekly virtual events where fellows will practice and reflect individually and in teams of their peers. LLM Red Teaming Coaches will be expected to prepare and facilitate live virtual events, including building supplementary slides and concept reviews based on SME-developed course content; monitor fellows' progress in Canvas; grade and provide feedback on portfolio deliverables; host virtual office hours, engage with fellows in Discord to answer questions and lead discussions; and model and reinforce ethical red teaming practices throughout. Each Coach will be responsible for up to 50 fellows, and will engage with the wider cohort of up to 500 fellows in shared settings such as Workshops, Maker Days, and the program Discord.

Requirements

Demonstrated aptitude for LLM red teaming, assessed through an exercise provided by Break Through Tech and a relevant work sample (e.g., a writeup, project, or portfolio piece).
Experience using large language models (LLMs) and leveraging advanced prompt engineering techniques.
Commitment to ethical, responsible red teaming practices, and the ability to model and uphold them with fellows.
Strong analytical and creative problem solving skills.
Excellent written, verbal, and interpersonal communication skills.
Experience evaluating and providing constructive feedback on technical work.
Strong organizational and time management skills, with the ability to work independently in a remote environment.
Experience planning and facilitating live events, workshops, or group discussions, ideally in a virtual setting and for an undergraduate student audience.
Commitment to developing inclusive and equitable working relationships with fellows from a variety of backgrounds.

Nice To Haves

Bachelor's or Associate's degree in computer science, cybersecurity, computational linguistics, or a related field, or equivalent practical experience.
Hands-on experience in ethical, sanctioned LLM red teaming challenges or competitions on platforms such as Gray Swan Arena, Lakera Agent Breaker, or HackAPrompt, including applying and iterating on common jailbreaking or prompt injection techniques.
Professional background in AI security, trust and safety, or ethical red teaming at a foundation AI lab, enterprise, or security firm.
Experience teaching, tutoring, TA-ing, or mentoring undergraduate students or early-career professionals.

Responsibilities

Design and facilitate concept reviews and hands-on practice activities for 4 Workshops (1.5 hours each) and 4 Maker Days (3 hours each), based on both a deep understanding of the fellow-facing course materials in Canvas, and a knowledge of current events and trends in the LLM red teaming / AI safety and security space.
Review all pre-scheduled red teaming challenges prior to the events to ensure that they are still active; test them out firsthand and document the process to complete them; and be able to guide the fellows through the same experience.
Create and monitor breakout rooms to ensure compliance with program requirements and ethical standards, address any fellow concerns, and provide feedback and suggestions to ensure fellow success and engagement.
Track attendance and ensure survey completion at the end of each event.
Review SME-developed course content in Canvas each week to stay ahead of fellows' learning and anticipate where they might need additional support.
Provide technical mentorship – both 1:1 with fellows and in small group office hours settings – on LLMs, red teaming, and jailbreaking concepts and topics.
Monitor and respond to fellows' curriculum-related questions in Discord.
Proactively reinforce ethics, responsible disclosure, and safety precautions throughout the program.
Provide written and oral feedback on fellows’ red teaming attempts, primarily completed through the Gray Swan Arena.
Find and share timely resources in the program's Discord, such as papers, news, new competitions, and notable jailbreak research, to keep fellows connected to the field.
Attend and actively engage in weekly Community of Practice meetings with other Break Through Tech Coaches and program staff, to share ideas and troubleshoot together.
Review assignments submitted by fellows in Canvas, providing written feedback where applicable and leveraging a grading rubric to score each assignment.
Review knowledge check activities in Canvas to identify areas where fellows are struggling with red teaming concepts; and use that information to create concept review and activity content for Workshops and Maker Days.
Partner with Break Through Tech program staff to identify any fellows who are failing to meet program expectations, and assist with supportive intervention and follow-up as needed.