Vice President, AI / ML Software Engineer

BNY Mellon•New York, NY

2d•Onsite

About The Position

Vice President AI/ML Software Engineer At BNY, our culture allows us to run our company better and enables employees’ growth and success. As a leading global financial services company at the heart of the global financial system, we influence nearly 20% of the world’s investible assets. Every day, our teams harness cutting-edge AI and breakthrough technologies to collaborate with clients, driving transformative solutions that redefine industries and uplift communities worldwide. Recognized as a top destination for innovators and champions of inclusion, BNY is where bold ideas meet advanced technology and exceptional talent. Together, we power the future of finance – and this is what #LifeAtBNY is all about. Join us and be part of something extraordinary. We are seeking a Vice President AI/ML Software Engineer to design and implement agentic AI systems, RAG pipelines, and intelligent document processing services. This is a senior individual contributor role with high autonomy -- you will own significant components of our AI platform, from embedding pipelines and vector retrieval to multi-agent extraction workflows. You will work closely with the SVP lead to translate architectural vision into production code, while mentoring mid-level engineers and driving technical excellence across the team. This role is in New York, NY

Requirements

Bachelor's degree in Computer Science, Engineering, or related field.
6+ years of professional software engineering experience
2+ years building production AI/ML systems (not just notebooks/prototypes.
Strong problem-solving skills with the ability to manage complex data processes.
Excellent collaboration and communication skills to work effectively with cross-functional teams.
Strong RAG expertise: Embedding models (OpenAI, sentence-transformers, Cohere, or similar), Vector databases (FAISS, Pinecone, Weaviate, Chroma, pgvector, or similar), Chunking and retrieval optimization, Context window management and prompt assembly
Agentic AI experience: Agent orchestration (custom frameworks, LangGraph, or similar), Tool-use patterns, function calling, structured output parsing, Memory and state management for multi-turn agent interactions
Python proficiency (3.11+): FastAPI, async patterns, Pydantic, Poetry, pytest
LLM integration: prompt engineering, token management, streaming, error handling, rate limiting
NLP & document processing: OCR post-processing, text segmentation, entity extraction
Testing rigor: unit tests, integration tests, golden-truth validation, retrieval metric evaluation
API design: RESTful services, OpenAPI specifications, versioning strategies

Nice To Haves

Advanced degree preferred.
Experience with knowledge graph construction from unstructured text
Familiarity with code AI concepts: code generation, automated testing, AI-assisted refactoring
Java/Spring Boot experience for cross-stack contribution
Angular/TypeScript for full-stack context
Experience with model evaluation: F1 scores, precision/recall for extraction, MRR/NDCG for retrieval
Exposure to fine-tuning or prompt optimization techniques
Understanding of graph RAG or hybrid retrieval architectures
Capital markets or financial services domain exposure
Experience with enterprise deployment: Docker, CI/CD, artifact repositories

Responsibilities

Implement agentic pipelines: agent loops, tool registries, memory stores, reasoning traces, and self-correction mechanisms
Build and optimize RAG systems end-to-end: Document ingestion and preprocessing (OCR output, PDFs, structured/unstructured text), Chunking strategies (section-aware, semantic, sliding window, hierarchical), Embedding generation and vector index management, Retrieval orchestration: hybrid search, metadata filtering, re-ranking, Context assembly and prompt construction for downstream LLM calls
Develop vectorization pipelines -- embedding model integration, batch processing, incremental index updates, and similarity search tuning
Implement multi-agent coordination patterns: shared blackboards, inter-agent messaging, task decomposition, and consensus mechanisms
Build prompt engineering infrastructure: template management, few-shot example selection, chain-of-thought scaffolding, and output parsing
Develop evaluation harnesses: automated accuracy measurement, retrieval quality metrics, regression detection, and A/B comparison tooling
Build FastAPI services exposing AI capabilities as production APIs (extraction, validation, classification)
Contribute to Java/Spring Boot platform services where AI integrates with business workflow
Design and maintain database schemas for AI metadata: audit trails, pipeline runs, memory entries, knowledge graphs
Implement content policy enforcement and data governance controls within AI pipelines
Mentor 2-3 mid-level engineers on AI engineering practices
Participate in architecture reviews and design sessions
Document patterns, decisions, and runbooks for AI system operation
Collaborate with product and business stakeholders to translate requirements into technical solutions