Senior Specialist, AI / Machine Learning Software Engineer

BNY Mellon•New York, NY

15h•$71,000 - $140,000•Onsite

About The Position

We are seeking a Senior Associate AI/ML Software Engineer to design and implement agentic AI systems, RAG pipelines, and intelligent document processing services. This is a senior individual contributor role with high autonomy -- you will own significant components of our AI platform, from embedding pipelines and vector retrieval to multi-agent extraction workflows. You will work closely with the SVP lead to translate architectural vision into production code, while mentoring mid-level engineers and driving technical excellence across the team. This role is in New York, NY. What Sets This Role Apart - You build the agent framework, not just configure one -- custom orchestration engine, not a LangChain wrapper - Production AI with real consequences -- extraction accuracy directly impacts financial operations - Full RAG ownership -- from raw OCR bytes through embedding, retrieval, and generation - Evaluation-driven culture -- golden-truth datasets, automated regression, measurable quality gates - Greenfield AI + enterprise integration -- build new AI-native systems that plug into established platforms. In this role, you'll have the opportunity to impact on our organization in the following ways: AI Systems Development Implement agentic pipelines: agent loops, tool registries, memory stores, reasoning traces, and self-correction mechanisms - Build and optimize RAG systems end-to-end: - Document ingestion and preprocessing (OCR output, PDFs, structured/unstructured text) - Chunking strategies (section-aware, semantic, sliding window, hierarchical) - Embedding generation and vector index management - Retrieval orchestration: hybrid search, metadata filtering, re-ranking - Context assembly and prompt construction for downstream LLM calls - Develop vectorization pipelines -- embedding model integration, batch processing, incremental index updates, and similarity search tuning - Implement multi-agent coordination patterns: shared blackboards, inter-agent messaging, task decomposition, and consensus mechanisms - Build prompt engineering infrastructure: template management, few-shot example selection, chain-of-thought scaffolding, and output parsing - Develop evaluation harnesses: automated accuracy measurement, retrieval quality metrics, regression detection, and A/B comparison tooling Platform & Backend Engineering Build FastAPI services exposing AI capabilities as production APIs (extraction, validation, classification) - Contribute to Java/Spring Boot platform services where AI integrates with business workflow - Design and maintain database schemas for AI metadata: audit trails, pipeline runs, memory entries, knowledge graphs - Implement content policy enforcement and data governance controls within AI pipelines

Requirements

Bachelor's degree in Computer Science, Engineering, or related field.
2-3 years of professional software engineering experience
2+ years building production AI/ML systems (not just notebooks/prototypes.
Strong problem-solving skills with the ability to manage complex data processes.
Excellent collaboration and communication skills to work effectively with cross-functional teams.
Strong RAG expertise: Embedding models (OpenAI, sentence-transformers, Cohere, or similar), Vector databases (FAISS, Pinecone, Weaviate, Chroma, pgvector, or similar), Chunking and retrieval optimization, Context window management and prompt assembly
Agentic AI experience: Agent orchestration (custom frameworks, LangGraph, or similar), Tool-use patterns, function calling, structured output parsing, Memory and state management for multi-turn agent interactions
Python proficiency (3.11+): FastAPI, async patterns, Pydantic, Poetry, pytest
LLM integration: prompt engineering, token management, streaming, error handling, rate limiting
NLP & document processing: OCR post-processing, text segmentation, entity extraction
Testing rigor: unit tests, integration tests, golden-truth validation, retrieval metric evaluation
API design: RESTful services, OpenAPI specifications, versioning strategies

Nice To Haves

Advanced degree preferred.
Experience with knowledge graph construction from unstructured text
Familiarity with code AI concepts: code generation, automated testing, AI-assisted refactoring
Java/Spring Boot experience for cross-stack contribution
Angular/TypeScript for full-stack context
Experience with model evaluation: F1 scores, precision/recall for extraction, MRR/NDCG for retrieval
Exposure to fine-tuning or prompt optimization techniques
Understanding of graph RAG or hybrid retrieval architectures
Capital markets or financial services domain exposure
Experience with enterprise deployment: Docker, CI/CD, artifact repositories

Responsibilities

Implement agentic pipelines: agent loops, tool registries, memory stores, reasoning traces, and self-correction mechanisms
Build and optimize RAG systems end-to-end: Document ingestion and preprocessing (OCR output, PDFs, structured/unstructured text), Chunking strategies (section-aware, semantic, sliding window, hierarchical), Embedding generation and vector index management, Retrieval orchestration: hybrid search, metadata filtering, re-ranking, Context assembly and prompt construction for downstream LLM calls
Develop vectorization pipelines -- embedding model integration, batch processing, incremental index updates, and similarity search tuning
Implement multi-agent coordination patterns: shared blackboards, inter-agent messaging, task decomposition, and consensus mechanisms
Build prompt engineering infrastructure: template management, few-shot example selection, chain-of-thought scaffolding, and output parsing
Develop evaluation harnesses: automated accuracy measurement, retrieval quality metrics, regression detection, and A/B comparison tooling
Build FastAPI services exposing AI capabilities as production APIs (extraction, validation, classification)
Contribute to Java/Spring Boot platform services where AI integrates with business workflow
Design and maintain database schemas for AI metadata: audit trails, pipeline runs, memory entries, knowledge graphs
Implement content policy enforcement and data governance controls within AI pipelines