Vice President, AI / ML Software Engineer
About The Position
We are seeking a Vice President AI/ML Software Engineer to design and implement agentic AI systems, RAG pipelines, and intelligent document processing services. This is a senior individual contributor role with high autonomy -- you will own significant components of our AI platform, from embedding pipelines and vector retrieval to multi-agent extraction workflows. You will work closely with the SVP lead to translate architectural vision into production code, while mentoring mid-level engineers and driving technical excellence across the team. This role is in New York, NY. What Sets This Role Apart - You build the agent framework, not just configure one -- custom orchestration engine, not a LangChain wrapper - Production AI with real consequences -- extraction accuracy directly impacts financial operations - Full RAG ownership -- from raw OCR bytes through embedding, retrieval, and generation - Evaluation-driven culture -- golden-truth datasets, automated regression, measurable quality gates - Greenfield AI + enterprise integration -- build new AI-native systems that plug into established platforms.
Requirements
- Bachelor's degree in Computer Science, Engineering, or related field.
- 6+ years of professional software engineering experience
- 2+ years building production AI/ML systems (not just notebooks/prototypes.
- Strong problem-solving skills with the ability to manage complex data processes.
- Excellent collaboration and communication skills to work effectively with cross-functional teams.
- Strong RAG expertise: Embedding models (OpenAI, sentence-transformers, Cohere, or similar)
- Vector databases (FAISS, Pinecone, Weaviate, Chroma, pgvector, or similar)
- Chunking and retrieval optimization
- Context window management and prompt assembly
- Agentic AI experience: Agent orchestration (custom frameworks, LangGraph, or similar)
- Tool-use patterns, function calling, structured output parsing
- Memory and state management for multi-turn agent interactions
- Python proficiency (3.11+): FastAPI, async patterns, Pydantic, Poetry, pytest
- LLM integration: prompt engineering, token management, streaming, error handling, rate limiting
- NLP & document processing: OCR post-processing, text segmentation, entity extraction
- Testing rigor: unit tests, integration tests, golden-truth validation, retrieval metric evaluation
- API design: RESTful services, OpenAPI specifications, versioning strategies
Nice To Haves
- Advanced degree preferred.
- Experience with knowledge graph construction from unstructured text
- Familiarity with code AI concepts: code generation, automated testing, AI-assisted refactoring
- Java/Spring Boot experience for cross-stack contribution
- Angular/TypeScript for full-stack context
- Experience with model evaluation: F1 scores, precision/recall for extraction, MRR/NDCG for retrieval
- Exposure to fine-tuning or prompt optimization techniques
- Understanding of graph RAG or hybrid retrieval architectures
- Capital markets or financial services domain exposure
- Experience with enterprise deployment: Docker, CI/CD, artifact repositories
Responsibilities
- Implement agentic pipelines: agent loops, tool registries, memory stores, reasoning traces, and self-correction mechanisms
- Build and optimize RAG systems end-to-end: Document ingestion and preprocessing (OCR output, PDFs, structured/unstructured text), Chunking strategies (section-aware, semantic, sliding window, hierarchical), Embedding generation and vector index management, Retrieval orchestration: hybrid search, metadata filtering, re-ranking, Context assembly and prompt construction for downstream LLM calls
- Develop vectorization pipelines -- embedding model integration, batch processing, incremental index updates, and similarity search tuning
- Implement multi-agent coordination patterns: shared blackboards, inter-agent messaging, task decomposition, and consensus mechanisms
- Build prompt engineering infrastructure: template management, few-shot example selection, chain-of-thought scaffolding, and output parsing
- Develop evaluation harnesses: automated accuracy measurement, retrieval quality metrics, regression detection, and A/B comparison tooling
- Build FastAPI services exposing AI capabilities as production APIs (extraction, validation, classification)
- Contribute to Java/Spring Boot platform services where AI integrates with business workflow
- Design and maintain database schemas for AI metadata: audit trails, pipeline runs, memory entries, knowledge graphs
- Implement content policy enforcement and data governance controls within AI pipelines
- Mentor 2-3 mid-level engineers on AI engineering practices
- Participate in architecture reviews and design sessions
- Document patterns, decisions, and runbooks for AI system operation
- Collaborate with product and business stakeholders to translate requirements into technical solutions
Benefits
- Highly competitive compensation
- Benefits and wellbeing programs
- Access to flexible global resources and tools
- Generous paid leaves, including paid volunteer time
- 401(k) plan
- Company-sponsored medical, dental, vision, and basic life insurance plans
- Various paid time off benefits, such as vacation and sick time
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
