Vice President - AI Safety Platform Engineering

Goldman Sachs•New York, NY

5d•$130,000 - $250,000•Onsite

About The Position

We are seeking a Vice President – AI Safety Platforms to build and lead our enterprise AI safety engineering initiatives. As generative AI in financial services evolves from simple prompt-response workflows to autonomous agentic systems that execute multi-step plans, call APIs, and interact directly with internal systems, establishing robust safety mechanisms and standardized evaluation protocols is essential. In this role, you will recruit and lead dedicated engineering pods focused on developing a unified company-wide agentic evaluation framework, real-time LLM guardrail services, and automated governance controls. As the senior technical authority for AI safety, you will collaborate closely with core AI platform teams, risk control functions, and business units to drive necessary enhancements to the core AI platform (such as telemetry hooks, API capabilities, execution sandboxes, and data logging infrastructure) to ensure all enterprise AI deployments operate safely, verifiably, and in compliance with institutional standards.

Requirements

Vice President experience (or equivalent senior engineering leadership) in financial services or large-scale enterprise software environments.
Bachelor’s or Master’s degree in Computer Science, Artificial Intelligence, Systems Engineering, or a related quantitative field.
4+ years leading applied ML or software engineering teams in building platform infrastructure or microservices.
8+ years of hands-on software development experience (Python, Go, Java, or C++) building microservices, high-throughput APIs, or enterprise platform services.
Technical fluency with Large Language Models (LLMs), RAG systems, function calling / tool integration, and agentic execution paradigms (e.g., LangChain, AutoGen, CrewAI, MCP server architectures).

Nice To Haves

Direct experience building agent evaluation frameworks and metrics (e.g., LLM-as-a-Judge, G-Eval, trajectory trace evaluation, task completion scoring).
Hands-on experience integrating low-latency guardrail tools and runtime filters (e.g., NeMo Guardrails, Guardrails AI, Llama Guard).
Experience with LLM and agent tracing tools (e.g., LangSmith, OpenTelemetry, Phoenix, MLflow) and structured audit logging infrastructure.
Proven ability to partner across teams and drive key governance capabilities into core shared platforms.

Responsibilities

Design, build, and deploy a single, company-wide agentic evaluation framework that standardizes how teams across all business lines benchmark, test, and measure AI agent performance prior to production deployment.
Implement evaluation methodologies that score autonomous planning quality, tool-calling precision, multi-turn state retention, trajectory efficiency, and error-recovery behaviors.
Integrate automated evaluation pipelines into runtime environments to continuously audit agent execution traces, detecting reasoning drift, tool failure modes, and unexpected trajectory shifts in production.
Establish standardized test suites and synthetic evaluation benchmarks tailored to complex financial workflows, such as automated research, risk assessment, and operational task execution.
Architect and scale enterprise guardrail microservices that inspect prompt inputs, retrieved context, and model outputs in real time to prevent data leakage, policy violations, and unvalidated execution.
Implement runtime policy gateways that inspect and authorize tool calls before execution, ensuring agents operate within authorized data boundaries and action scopes.
Build configurable escalation workflows and approval gates that automatically pause execution for high-risk operations (e.g., money movement, client record modifications, or external communications) until human authorization is granted.
Partner directly with the core AI Platform team to drive the implementation of safety APIs, telemetry hooks, developer SDKs, and MLOps/LLMOps pipeline integrations.
Define and enforce technical standards for immutable audit logging, execution tracing (e.g., OpenTelemetry standards), and principal identity propagation across all agentic workflows.
Translate model risk management standards (e.g., SR 11-7 / SR 26-2 guidance, FINRA supervision requirements) into automated engineering safeguards and policy checks.
Hire, develop, and mentor high-performing engineering teams specializing in applied machine learning, AI safety, and enterprise platform engineering.
Own the technical roadmap for enterprise AI safety infrastructure, setting clear milestones for evaluation framework adoption, runtime latency optimization, and governance automation.
Articulate technical risk profiles, evaluation metrics, and safety architecture to risk committees, model validation teams, and executive leadership.