Principal Researcher, Benchmarks, Evaluation, and Standards

Partnership on AI

13h • $133,900 - $197,800 • Remote

About The Position

The Partnership on AI (PAI) seeks a Principal Researcher to lead the technical backbone of its work, with emphasis on benchmarking, evaluation, and technical standards. Reporting to the Chief Programs and Insights Officer, the role works closely with leaders across PAI’s focus areas: AI, Human Connection, and Well-Being; Trustworthy AI Value Chains; and AI & Shared Prosperity. The position directs how PAI brings its sociotechnical philosophy into practice, treating technical and social considerations as connected problems. It ensures PAI’s guidance across model, product, and policy layers is grounded in the public interest and stakeholder input, technically implementable and measurable, and transferred to appropriate organizations responsible for benchmarks, evaluations, and standards.

The Principal Researcher develops PAI’s strategy and perspective on the evaluation and benchmarking ecosystem; provides technical leadership across focus areas and teams; and helps turn PAI’s guidance into evaluations, benchmarks, standards, testing, and sandboxing exercises. The role also sustains partnerships and quality control, produces research and policy publications, convenes and represents PAI with diverse stakeholders, and translates findings into clear recommendations for industry, policymakers, civil society, and affected communities. It collaborates with the Partnerships team on fundraising and potential earned-revenue approaches.

This is a remote-first role eligible for work from the United States, Canada, London, UK, or Brussels, Belgium. Candidates must be authorized to work in their country of residence; visa sponsorship is unavailable. Compensation varies by location: United States $133,900–$197,800 USD; Canada C$170,200–C$204,000 CAD; London £109,100–£130,800 GBP; Brussels €135,600–€162,600 EUR. In Belgium, the required 13th-month pay and double holiday pay are additional to the base salary range. Applications are reviewed on a rolling basis until the position is filled; applicants should submit a cover letter and resume. The requested cover letter should be no more than 750 words and address how normative guidance can be better integrated with technical approaches to support impact and adoption by AI developers and deployers.

Requirements

• Eight or more years of relevant technical experience across some combination of AI, human-computer interaction, trust and safety, or another relevant interdisciplinary field.
• Fluency in technical and sociotechnical domains; priority is given to candidates with advanced degrees in a technical discipline such as computer science.
• Strong, accessible writing for varied audiences; written outputs must be compelling enough to support adoption by other organizations.
• Experience translating guidance into evaluations, benchmarks, standards, audit criteria, or input to certifications and professional training.
• Demonstrated ability to interface with and convene stakeholders with conflicting interests and varied disciplinary training, and produce outcomes the group can support.
• Experience representing and speaking on behalf of an organization at events featuring executives and leaders from varied domains and backgrounds.
• Demonstrated ability to set objectives independently and drive multistakeholder strategy with a high degree of autonomy.
• Authorization to work in the country where the candidate will be based; PAI cannot sponsor visas.

Nice To Haves

• PhD in computer science, AI, or a comparably technical discipline.
• Experience actively participating in standards development and working with implementation partners.
• Professional emphasis on technology’s social impact and experience applying technical skills to social questions.
• Familiarity with how AI systems are built and evaluated, including model behavior specification, red-teaming, and automated evaluation.
• Experience with questions related to psychological well-being or content authenticity.
• Comfort with domestic and international travel, which may at times be substantial.
• Comfort working in a small, evolving, cross-disciplinary organization and contributing to priorities beyond one’s immediate focus.

Responsibilities

• Develop PAI’s perspective on the evaluation and benchmarking ecosystem, assess infrastructure needed for PAI’s evaluations, and direct its strategy for engaging with that ecosystem in collaboration with the supervisor and executive team.
• Advance adoption of shared industry guidance through development of and integration into benchmarks, evaluations, and technical standards, in partnership with relevant auditing and certification organizations.
• Provide technical leadership and help shape the strategic direction of PAI’s focus areas in collaboration with their leads and the Chief Programs and Insights Officer.
• Support technical efforts across other PAI teams, including Policy and International.
• Work with the Partnerships team to structure work that supports fundraising.
• Identify urgent sociotechnical questions PAI is well placed to address and design implementation pathways.
• Ensure guidance across PAI projects is technically feasible and implementable, including as evaluations or standards, and support relevant testing and sandboxing exercises.
• Design an approach to maintaining PAI’s benchmarking, evaluation, and standards work over time, including quality control for established partnerships.
• Author or co-author peer-reviewed research papers, policy briefs, and accessible reports that translate PAI insights into actionable recommendations.
• Initiate and lead collaborative research partnerships with academic institutions, civil society organizations, and industry partners.
• Represent PAI at selected conferences and other venues, presenting findings and methodological innovations.
• Lead key relationships with benchmarking, technical standards, and evaluation-oriented organizations and settings, including AISIs and assurance bodies.
• Collaborate with the Partnerships team to test earned-revenue models, including licensing or content contributions to credentialing and training programs.
• Translate rigorous findings into clear narratives for industry and civil society leaders, policymakers, and affected communities.
• Support strategic communications to amplify and increase adoption of PAI’s guidance.
• Represent PAI’s work to standards bodies, policymakers, media, funders, and the public, and connect open research questions to the resources needed to answer them.

Benefits

• Twenty vacation days, three personal reflection days, and sick and family leave above industry standards.
• High-quality PPO and HMO health insurance plans, many 100% covered by PAI.
• Dental and vision insurance, 100% covered by PAI.
• 401(k) match of up to 7%, vested immediately.
• Automatic cell phone reimbursement of $75 per month.
• Annual funds for professional development.
• $150 per month for co-working space access.
• Regular team lunches and focused work days.
• Opportunities to attend AI-related conferences and collaborate with partners across industry, academia, and civil society.

What This Job Offers

• Job Type: Full-time
• Career Level: Principal
• Education Level: No Education Listed