AI Product Engineer

BDIPlus•New York, NY

5d•$100,000 - $120,000

About The Position

As an AI Product Engineer at BDIPlus, you will be responsible for designing, developing, and delivering scalable enterprise software products that incorporate modern AI capabilities. This is a hands-on product development role requiring extensive full stack engineering experience, strong system design skills, and the ability to build products from architecture through production. You will work across the full technology stack, developing user-facing functionality, backend services, APIs, data integrations, and platform components while incorporating AI capabilities such as agents, agentic workflows, RAG, LLM pipelines, embeddings, and vector search where they add value to the product. You will work closely with Product, Engineering, Data Science, and Infrastructure teams to translate product requirements into well-designed, scalable software solutions. We are looking for an engineer who thinks beyond individual AI use cases. The ideal candidate understands how to build products, make sound architectural decisions, develop across the full stack, and integrate AI and agentic capabilities into reliable enterprise software.

Requirements

Master's degree or equivalent experience required; PhD a plus.
3+ years of professional software engineering, product engineering, or related development experience.
Extensive hands-on full stack development experience, with demonstrated responsibility for developing production software products.
Strong experience building modern front-end applications and backend services.
Strong system design and software architecture capabilities.
Demonstrated ability to design and build scalable, distributed, production-grade systems.
Strong Python development skills.
Experience with JavaScript/TypeScript and modern front-end frameworks such as React or Next.js.
Strong experience developing APIs and backend services using technologies such as FastAPI or Flask.
Strong SQL skills and experience working with relational and/or NoSQL databases.
Hands-on experience implementing agents and agentic workflows.
Experience with LangGraph, LangChain, or comparable orchestration frameworks.
Experience designing and implementing RAG and LLM pipelines.
Strong understanding of embeddings, vector embeddings, vector databases, semantic search, and similarity search.
Experience with retrieval techniques including chunking, metadata enrichment, vector search, keyword search, hybrid search, re-ranking, and retrieval optimization.
Experience integrating applications with external APIs, databases, enterprise systems, and data platforms.
Strong understanding of software engineering fundamentals, design patterns, microservices, distributed systems, and API design.
Experience with Git, CI/CD, automated testing, and modern software development practices.
Ability to independently own technical problems and product features from design through production.

Nice To Haves

Experience building enterprise AI-enabled software products.
Experience designing complex workflows involving multiple agents, tools, APIs, and data sources.
Experience with LangGraph state management, tool calling, routing, persistence, memory, and human-in-the-loop workflows.
Experience with vector platforms such as Elasticsearch/OpenSearch Vector, FAISS, Pinecone, Weaviate, Milvus, or pgvector.
Experience with LLM evaluation, observability, prompt management, and model monitoring.
Experience with AWS, Azure, or GCP.
Docker and Kubernetes experience.
Experience with Snowflake, Databricks, or other enterprise data platforms.
Experience developing software within regulated or security-conscious enterprise environments.
Financial Services or Insurance industry experience.

Responsibilities

Design, develop, and deliver enterprise software products and product features end-to-end.
Take ownership of product development from technical design and architecture through implementation, testing, deployment, and ongoing enhancement.
Build modern front-end experiences, backend services, APIs, data integrations, and platform components.
Translate product requirements and user needs into scalable technical solutions.
Develop reusable services and components that support multiple product capabilities and workflows.
Work closely with Product teams to evaluate requirements, technical feasibility, tradeoffs, and implementation approaches.
Apply strong software engineering practices including modular architecture, automated testing, CI/CD, code reviews, and secure development.
Own and contribute to system design and technical architecture for new products, features, and platform capabilities.
Design scalable, distributed systems that support enterprise workloads.
Define service boundaries, APIs, data flows, integration patterns, and application architecture.
Make architectural decisions considering scalability, latency, resiliency, security, maintainability, observability, and cost.
Design systems capable of integrating traditional application logic, enterprise data, and AI-driven functionality.
Clearly document architecture decisions and technical tradeoffs.
Design and implement AI agents and agentic workflows as integrated capabilities within BDIPlus products.
Build stateful, multi-step workflows using LangGraph and related orchestration frameworks.
Implement tool calling, routing, memory, state management, human-in-the-loop processes, and workflow orchestration.
Enable agents to securely interact with APIs, databases, enterprise systems, and product services.
Determine when agentic approaches are appropriate versus deterministic workflows and traditional application logic.
Design agent workflows with appropriate controls, monitoring, fallbacks, and error handling for production environments.
Design and implement RAG and LLM pipelines that support product functionality.
Build ingestion and processing pipelines for structured and unstructured enterprise information.
Implement document processing, chunking, metadata enrichment, embeddings/vector embeddings, indexing, and retrieval.
Design and optimize semantic search, vector search, keyword search, and hybrid search.
Implement filtering, re-ranking, query rewriting, context optimization, and retrieval evaluation.
Integrate LLMs and retrieval capabilities into broader application and product workflows.
Implement grounding, citations, validation, and hallucination-mitigation techniques.
Build permission-aware retrieval aligned with enterprise access controls and entitlements.
Build software that is secure, scalable, testable, observable, and production-ready.
Establish automated testing and evaluation across traditional software components and AI-powered functionality.
Monitor application performance, latency, errors, token usage, model behavior, and infrastructure costs.
Implement caching, rate limiting, retries, fallbacks, and resilient error handling.
Troubleshoot issues across the full stack, including application, API, data, retrieval, LLM, and infrastructure layers.
Continuously improve existing product capabilities based on performance, usage, and evolving requirements.
Work closely with Product Managers and business stakeholders throughout the product development lifecycle.
Partner with Data Science, Data Engineering, Infrastructure, Security, and UX teams to deliver integrated product capabilities.
Participate in product and technical design discussions and contribute ideas for new capabilities and improvements.
Communicate complex technical concepts and architecture decisions clearly to technical and non-technical stakeholders.
Produce high-quality technical documentation, system diagrams, and implementation guidance.