Senior RAG Engineer

Newcode.ai • New York, NY

3d • Remote

About The Position

Newcode.ai is a fast-growing legal tech and agentic AI company transforming how legal work is done. With teams across Norway, Sweden, Ireland, the US and growing we work at the intersection of law, technology, and intelligence. We move fast, think big, and take pride in doing things the right way. Note: We believe in being transparent about what it's like to work at Newcode. As a fast-growing startup, we're building and evolving every day. That means not every process, playbook, or framework is already in place, and priorities can shift quickly. The people who thrive here are comfortable with ambiguity, take ownership, and don't wait for perfect direction. They are resourceful, proactive, and able to "figure it out"—solving problems, creating structure where needed, and helping build the company as they go. If you are good with this then, great! Keep reading to learn more.

You'll own retrieval. When someone asks our product a question, something has to find the right passage across large sets of confidential documents, decide it really is the right passage, and hand it to a language model whose answer ends up in real professional advice. That whole path is yours: how documents get parsed, chunked and embedded, how search combines meaning with exact terms, how results get reranked, and how an agent decides what to read next and when it has read enough. This is where AI products fail without anyone noticing. The answer reads well, the citation is wrong, and the client is the one who finds out.

It isn't CRUD work, and it isn't a demo notebook either. Retrieval is where AI products quietly fail: the answer reads well, the citation is wrong, and nobody notices until a client does. Your job is to stop that happening, and to be able to show with numbers that it isn't happening. You'll make architecture calls early and live with them.
The role is fully remote. Work from anywhere in the EU.

Requirements

• At least five years building backend systems that run in production, and at least two of them shipping retrieval or RAG systems real users depend on.
• You've diagnosed a retrieval regression in production and fixed it. You can tell us what broke, how you found it, and what the numbers were before and after.
• You've built and maintained a golden set. How many queries, who labelled them, which metrics you trust, and a change you shipped or killed because of what they told you.
• You've run a vector index in production: picked index parameters, dealt with the memory and latency trade-offs, and reindexed without taking search down.
• Strong Python.
• Comfortable reasoning about FastAPI or a close equivalent, PostgreSQL and Redis under load.
• You don't ship a retrieval change because the output looked better on the three queries you tried by hand.
• You own things end to end — including the boring maintenance and the tech debt nobody assigned you — rather than building the interesting part and handing off the rest.
• You track what's moving in vector databases and LLMs because you're curious, not because it's the job. Show us the side project, the benchmark you ran for fun, or the repo where you tried something before it showed up in everyone else's stack.
• You've worked with AI coding assistants and have a view on where they help and where they don't. If your current employer forbids them, that's not a mark against you. Tell us how you'd review AI-written code instead.
• You're fine in a startup that changes direction. Decisions get made, then revisited.
• Python: 5+ years
• FastAPI: 2+ years
• Vector databases in production (we run Qdrant): 2+ years. You've picked index parameters, dealt with the memory and latency trade-offs, and reindexed without taking search down.
• Embedding models, and the metrics you use to judge retrieval quality
• PostgreSQL / SQL
• OCR-heavy document ingestion at scale; an information-retrieval background (BM25, NDCG, recall@k); search abstractions spanning more than one backend.

Nice To Haves

• Backend depth without production retrieval won't be enough for this role, but our Senior Backend Engineer role might be the better fit.

Responsibilities

• Own the retrieval pipeline end to end: document parsing and chunking, embeddings, indexing, and keeping all of it current as clients add and change material. Exposed through FastAPI, with ingestion and reindexing running as background jobs.
• Design how we search. Combining semantic and keyword retrieval in Qdrant, fusing ranked lists, filtering on metadata, and reranking so the handful of results we pass to the model are the right ones.
• Build agentic retrieval: query decomposition, the tools a model uses to search and navigate documents, multi-step loops that know when to stop, and the cost and latency budgets that keep them honest.
• Build the evaluation layer that tells us any of this is working: golden sets, retrieval metrics, regression tests on realistic client data, and tracing good enough that a bad answer leads you back to the chunk that caused it.
• Ship it as fast, dependable services in Python and FastAPI, with the heavy work - ingestion, embedding, reindexing - running as background jobs that hold up under load.
• Take data security and isolation seriously. Clients hand us privileged material, and tenants stay strictly separated.
• Make retrieval hold up across our clients' languages as well as it does in English. Compounding and inflection break sparse retrieval in ways an English eval set never shows you.

Benefits

• Fully remote across the EEA, Norway included.
• We employ through our own entity or a local employer of record depending on where you are, so ask about your country and we'll tell you straight away whether we can do it.
• Be a driver of innovation in one of the most exciting AI ventures
• Work with talented peers in a collaborative, high-energy team
• Shape both product and culture as we grow
• Flexible, English-speaking environment

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed