Research Engineer, Index Intelligence

Exa • San Francisco, CA

23h • $180,000 - $350,000

About The Position

Exa is an applied AI lab building a search engine unlike the world has ever seen. We build massive-scale infra to crawl the entire web, train state-of-the-art embedding models to process it, and design super high performant vector databases to retrieve over it. We now power search for Cursor, Cognition, HubSpot, and over 400,000 developers and have raised $350m from Lightspeed, Benchmark, and a16z. Our ultimate goal is to build perfect search over all the world's information, far beyond Google. If you want to build massive-scale ML systems that will define the way the new AI world consumes information, this is the place for you. We crawl the web continuously and cannot keep all of it, refresh all of it, or spend a GPU on all of it. Something has to decide which pages are worth indexing, which links are worth following, which pages say the same thing as pages we already have, what has gone stale, and what we should never have picked up. These decisions are worth more than most ranking work. The cheapest way to improve search is to stop indexing pages nobody should ever retrieve, and to start indexing the ones we are missing. Today they run on a mix of learned signals and thresholds someone picked. We are looking for a research engineer to own the signals behind those decisions, and the decisions themselves.

Requirements

• Comfortable owning a problem with no ground truth, where the first job is defining what the right answer means.
• Hands-on ML experience with classifiers, rankers and calibration.
• Strong data instincts at web scale.
• Think in end-to-end impact; a signal only counts if a decision changes and search gets better.
• Enjoy working across teams.
• Care about the problem of finding high quality knowledge and recognize how important this is for the world.

Responsibilities

• Own the signals behind decisions about web page indexing and link following.
• Own the decisions themselves regarding which pages are worth indexing, which links are worth following, which pages say the same thing as pages we already have, what has gone stale, and what we should never have picked up.
• Improve search by stopping indexing pages nobody should ever retrieve and starting indexing the ones we are missing.
• Work across teams, including crawling, indexing, and retrieval, as they consume what you build.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed