AI Engineer

Charity Navigator•Union City, NJ

1d•Remote

About The Position

Charity Navigator is seeking an AI Engineer to build and ship production LLM-powered and agentic features. This role focuses on application-layer work, partnering with the existing engineering team to make nonprofit profiles easier to search, compare, and understand. The AI Engineer will report to the VP of Engineering and work closely with them and other senior engineers on system design, technical direction, and strategic AI opportunities. The role involves owning AI features from day one, with opportunities for increased architectural ownership as experience grows. The goal is to provide donors with clear answers rather than just data.

Requirements

2-3+ years of experience building or deploying machine learning models or AI-driven applications.
Bachelor’s degree in Computer Science, Engineering, AI/ML, or a related field, or a relevant certificate program. Demonstrated technical proficiency and relevant industry experience will be considered in lieu of a formal degree.
Python and ML fundamentals.
2+ years of Python professional or substantial project-based experience, with a solid grasp of core machine learning concepts and API integration.
Hands-on experience with foundation model providers (Anthropic, OpenAI, and similar).
Experience with prompt engineering and agentic patterns, using orchestration frameworks like LangChain or LlamaIndex, or integration standards like the Model Context Protocol.
Practical experience querying and tuning a vector database (pgvector, OpenSearch, or similar) for RAG applications.

Nice To Haves

Managed cloud AI services, such as Amazon Bedrock.
Cloud deployment of application code via serverless or containerized patterns, AWS required.
Scrum or other Agile development experience.
Cloud, Gen AI, or machine learning certifications, such as AWS Certified Generative AI Developer, Certified Machine Learning, or Solutions Architect.

Responsibilities

Build and ship production LLM-powered and agentic features, including prompt orchestration, tool calling, embeddings, and structured outputs, favoring reusable components.
Build API Integrations, MCP servers, and retrieval-augmented generation (RAG) workflows to connect existing data systems to vector search for accurate surfacing and explanation of ratings data, with source citation.
Build and run evaluation infrastructure, including prompt regression tests, LLM-as-a-judge evaluations, and synthetic datasets, to proactively identify feature quality regressions.
Monitor AI features in production, tracking accuracy, latency, token cost, and hallucination rates, and debug/tune prompts accordingly.
Implement runtime safety and reliability guardrails for AI outputs to maintain quality, compliance, and operational bounds.
Research and recommend orchestration frameworks, vector stores, and model providers, considering cost and operational burden against speed, with joint build-versus-buy decisions with the VP of Engineering.
Work directly with non-technical stakeholders to scope problems and provide realistic assessments of AI capabilities.
Evaluate emerging models and tools, utilize AI-assisted development extensively, and improve team engineering standards through code review.