Senior Vice President, AI / Machine Learning Software Engineer

BNY Mellon•New York, NY

1d•$120,000 - $200,000•Onsite

About The Position

At BNY, our culture allows us to run our company better and enables employees’ growth and success. As a leading global financial services company at the heart of the global financial system, we influence nearly 20% of the world’s investible assets. Every day, our teams harness cutting-edge AI and breakthrough technologies to collaborate with clients, driving transformative solutions that redefine industries and uplift communities worldwide. Recognized as a top destination for innovators, BNY is where bold ideas meet advanced technology and exceptional talent. Together, we power the future of finance – and this is what #LifeAtBNY is all about. Join us and be part of something extraordinary. We are seeking a senior‑level engineer to design, build, and operate production‑grade GenAI and Retrieval‑Augmented Generation (RAG) platforms at scale. This role focuses on industrializing LLM‑based systems with strong guardrails, observability, evaluation frameworks, and operational rigor, ensuring reliability, safety, and cost efficiency across the full AI lifecycle.

Requirements

Strong experience building and operating production ML or GenAI systems in enterprise environments.
Deep hands‑on expertise with LLM orchestration frameworks, such as LangChain and/or LlamaIndex.
Experience with model registries and experiment tracking, such as MLflow or equivalent.
Solid understanding of Kubernetes‑based deployments and cloud‑native architectures.
Familiarity with feature stores, data pipelines, and retriever/index lifecycle management.
Proven experience implementing telemetry, logging, metrics, and distributed tracing for ML/AI workloads.
Strong knowledge of CI/CD practices for ML, GenAI, and data‑driven systems.

Nice To Haves

Experience operating LLM systems at scale, including multi‑model or multi‑provider strategies.
Exposure to AI safety, governance, and compliance frameworks in regulated environments.
Background in SRE, platform engineering, or MLOps, with a reliability‑first mindset.
Ability to translate ambiguous GenAI use cases into robust, production‑grade architectures.

Responsibilities

Design and build production‑ready RAG pipelines, including retrieval, ranking, prompt orchestration, and response generation, with comprehensive guardrails, tracing, and observability.
Implement offline and online evaluation frameworks for prompts, models, and datasets, including quality, safety, latency, and cost metrics.
Own end‑to‑end lifecycle management for GenAI systems, covering prompt versions, model versions, datasets, and configurations.
Establish and maintain CI/CD pipelines for prompts, models, and data, enabling safe, repeatable, and auditable releases.
Implement cost and performance monitoring, including token usage, inference latency, throughput, and spend optimization.
Build and enforce safety mechanisms, such as content filtering, policy enforcement, red‑teaming feedback loops, and abuse detection.
Define and operationalize incident management workflows, including alerting, triage, rollback mechanisms, and post‑incident analysis.
Partner closely with product, platform, and governance teams to ensure GenAI solutions meet enterprise reliability, security, and compliance standards.
Mentor engineers and influence best practices for building scalable, trustworthy AI systems.