Director of Engineering, IAEF

Digital Harbor Foundation•Baltimore, MD

12d•$215,000 - $315,000•Remote

About The Position

The Founding Vice President, Engineering, leads the architecture, delivery, and engineering team for Independent AI Evaluation Foundation (IAEF), a platform for evaluating the trustworthiness of artificial intelligence (AI). Reporting to the CEO of IAEF, this leader combines technical ownership, team development, security leadership, and external representation. Independent AI Evaluation Foundation (IAEF)’s mission is to identify and reduce barriers to evaluating the efficacy, transparency, and safety of specific educational AI-centric solutions used by students and opportunity seekers in their educational context. IAEF is a fiscally sponsored project of Digital Harbor Foundation. Digital Harbor Foundation is dedicated to fostering learning, creativity, productivity, and community through education with a vision of digital equity for everyone. Driven by our sincere belief that access to opportunity is a basic right, we take bold yet practical actions to support making a better future now. Through a portfolio of projects focused on developing leadership within communities, we support those closest to challenges to take deliberate actions based in design thinking approach, backed by data analysis, grounded in a practice of collective impact, and driven by a commitment to racial equity.

Requirements

12+ years of software engineering experience, including at least four years in senior technical leadership at a startup, evaluation or audit firm, machine-learning infrastructure company, or research-grade platform team.
Experience building and operating a platform with a plugin, extension, or component model, and running AI evaluation, red-teaming, machine-learning observability, or model-testing systems in production.
A record of hiring and retaining senior engineers in a distributed team, including at least one applied machine-learning or agentic-systems specialist.
Ability to design reusable platform components and work with human-in-the-loop (HITL) transcripts, benchmark scores, and large language model (LLM) red-teaming results.
Ability to communicate technical decisions to enterprise security leaders, foundation program officers, investors, customers, and partners.
Ability to prioritize competing technical and organizational needs and lead a distributed engineering team through delivery, operational issues, and growth.
Understanding of the operational separation required between a public benefit corporation, a companion nonprofit, and grant-funded programs.

Nice To Haves

Prior CTO, VP Engineering, or founding-engineer experience at an early-stage company that raised institutional capital; experience enabling external developers to build on a platform; and prior evaluation-methodology work with academic or civil-society partners.
A PhD and extensive experience at a frontier AI research laboratory are not required.
Experience leading implementation of ISO 42001, the NIST AI Risk Management Framework, or SOC 2 requirements, rather than participating only as a reviewer.

Responsibilities

Own the Workbench’s Design → Test → Analyze → Publish (DTAP) workflow, Component Contract, Artifacts API, component registry, and canvas interface.
Create a persistent, structured, queryable evaluation history rather than one-off engagement exports, and integrate the existing red-teaming platform, administrative dashboard, and HITL data-capture architecture instead of rebuilding them from scratch.
Decide when to build or buy authentication, storage, orchestration, LLM-provider services, and sandboxing for the planned notebook environment.
In conjunction with the Human Resources Department of Digital Harbor Foundation, recruit, develop, and retain the engineering organization against the approved build plan, including Staff/Senior Backend, Senior Frontend, DevOps/site reliability, Applied Machine Learning, and Senior Product Manager roles
Set planning, code-review, on-call, incident-response, and roadmap-review practices; provide engineering progress and risks for board and funder reporting.
Partner with the IAFE’s CEO and Evaluation Science Lead to translate HITL red teaming, automated benchmarks, LLM red teaming, LLM-as-judge, and drift detection into a Component Contract that supports all three Test artifact types and planned agentic components.
Lead vendor risk reviews for LLM providers, cloud infrastructure, and third-party evaluation tools.
Coordinate with Digital Harbor Foundation’s Leadership, where applicable, on required approvals, funder restrictions, and separation of PBC and nonprofit resources, data, and intellectual property under approved agreements. Escalate unresolved risks rather than independently determining legal or sponsor policy.
Represent IAEF in technical conversations with customers, partners, funders, and press, and own the technical content of investor fundraising decks, roadmap materials, demonstrations, and due diligence reviews in partnership with the Executive Director of IAEF and Digital Harbor Foundation’s leadership.
Lead the developer-relations narrative for software and agent development kits (SDK/ADK) as the Component Contract opens to outside developers in Phase 1, partnering with the Developer Relations & Ecosystem hire planned for Period 2.

Benefits

Health, Dental, and Vision Insurance (100% of the premium paid for employees and 85% of dependents)
Pre-Tax Health Savings Account (HSA) (with $365 monthly employer contributions)
Pre-Tax Flexible Savings Account (FSA)
Paid Accidental Death & Dismemberment (AD&D) Insurance
Paid Short-Term & Long-Term Disability Insurance
Paid Basic Life Insurance
Supplemental Voluntary Life Insurance (Employee, Spouse & Dependent Children)
Total Pet Plan and Supplemental Wishbone Pet Insurance
Employee Opportunity Program (EAP) - Health and Wellness
Wellness Reimbursement Program
401k Retirement Plan (with 6% matching)
15 Days Paid Time Off Per Year
20 Days Paid Time Off Per Year (after 3rd Anniversary)
25 Days Paid Time Off Per Year (after 6th Anniversary)
16 Paid Holidays (14 common plus 2 flexible holidays, including Dec 25 - Jan 1)
Paid Bereavement Leave
Paid Parental Leave for Moms and Dads (two weeks after first year)