Senior Specialist / Manager, AI Safety Data and Evaluation Program

Lila Sciences • Cambridge, MA

1d • $88,000 - $144,000

About The Position

Lila Sciences is building a high-agency AI safety team to work across science, model training, lab integration, and other core teams. The team will identify and mitigate risks from scientific superintelligence while preserving the scientific discovery capabilities of Lila’s systems. Its initial work includes developing a safety strategy tailored to Lila’s goals and deployment plans, alongside technical safety strategy, ecosystem engagement, safety evaluations, risk-mitigation systems, and research into longer-term needs such as oversight of superintelligent scientific systems.

In this role, you will turn scientific risk questions into expert-data and evaluation programs that produce high-quality, interpretable evidence. You will connect internal risk mapping and threat modeling with expertise from Lila colleagues and external specialists. Your primary focus will be program execution and data quality: coordinating contributors, running red-teaming and evaluation campaigns, and ensuring outputs meet agreed standards. You will also gather targeted feedback from AI Safety community partners. Working with scientists who define research questions and interpret their significance, you will create the operational conditions for their expertise to inform safety decisions made by Lila’s senior leadership.

Requirements

• Experience delivering expert-data, research-operations, evaluation, or similarly demanding programs involving specialist contributors.
• A substantial working understanding of AI safety, including how threat models, capability evaluations, red teaming, and safeguards inform one another.
• Ability to interpret safety research briefs, recognize assumptions and the consequences of data-quality failures, and identify when scientific escalation is needed.
• Evidence of applying AI safety understanding in research, evaluation, or expert-data work, including how it shaped program scoping, execution, or review.
• Strong operational judgment and ability to manage dependencies, changing requirements, and multiple parallel workstreams.
• Track record of setting measurable quality standards, detecting recurring problems, and improving data-production processes.
• Clear communication, careful documentation, and ability to build effective relationships with researchers, experts, and vendors.
• Sufficient technical and scientific fluency to understand research briefs, clarify ambiguity, and work productively with subject-matter experts.

Nice To Haves

• Experience contributing to scientific study design, assessing scientific validity, or developing evaluation rubrics.
• Deeper scientific expertise or research experience in biology, chemistry, or materials science.
• Data analysis, scripting, annotation platforms, or tools for tracking research workflows and data provenance.

Responsibilities

• Translate agreed scientific briefs into executable plans covering tasks, contributor needs, instructions, milestones, quality standards, and delivery expectations.
• Source and coordinate external experts with scientific colleagues, and organize access to relevant expertise within Lila.
• Manage contributor onboarding, task assignment, training, and support.
• Run expert-data collection, annotation, red-teaming, and evaluation programs; track progress and resolve delivery problems.
• Establish quality controls for completeness, consistency, provenance, and rubric adherence.
• Coordinate calibration, review, disagreement resolution, and rework with scientific owners.
• Maintain clear records, versioned guidance, and usable datasets; improve workflows based on quality findings and researcher feedback.
• Identify questions requiring scientific judgment and bring them to appropriate specialists, ensuring decisions are reflected in program materials and subsequent work.
• Gather targeted high-impact feedback from AI Safety community partners on aspects of Lila’s safety approach.

Benefits

• Competitive base compensation with bonus potential and generous early-stage equity.
• Medical, dental, and vision coverage for full-time U.S. employees.
• Employer-paid life and disability insurance for full-time U.S. employees.
• Flexible time off and generous company-wide holidays.
• Paid parental leave.
• Educational assistance program.
• Commuter benefits, including bike share memberships for office-based employees.
• Company-subsidized lunch program.
• Regionally tailored comprehensive benefits for full-time employees outside the U.S.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed