Research Engineer (Focused on Search/IR)

Firecrawl•San Francisco, CA

4d•$180,000 - $270,000•Remote

About The Position

You'll own and advance the search and information retrieval systems at the core of Firecrawl — the infrastructure that determines how we find, rank, index, and serve web content at scale. This is a hands-on, full-stack search role where you'll build and operate everything from ingestion pipelines to serving layers. If you've built search indexes at massive scale and care deeply about ranking quality, freshness, and retrieval speed, this is the role. About Firecrawl Firecrawl is the easiest way to extract data from the web. Developers use us to reliably convert URLs into LLM-ready markdown or structured data with a single API call. In just a year, we've hit 8 figures in ARR and 90k+ GitHub stars by building the fastest way for developers to get LLM-ready data. We're a small, fast-moving, technical team building essential infrastructure super-intelligence will use to gather data on the web. We ship fast and deep.

Requirements

Someone who has built search indexes at massive scale. Not a tutorial project — real indexes serving real traffic with real latency requirements. You've dealt with the hard problems: sharding strategies, index compaction, schema evolution, and the operational complexity of keeping billions of documents queryable and fast.
Hands-on with ranking, relevance, and query understanding. You've built or meaningfully improved ranking systems. You understand BM25, learned ranking, embedding-based retrieval, and when to use which. You can reason about relevance tradeoffs and you've shipped ranking changes that moved metrics in production.
Owns the full stack: ingestion → index → serving. You're not a specialist who only touches one layer. You've built and operated the entire search pipeline — from how documents enter the system to how results get served. You understand the dependencies between layers and you make good architectural decisions because you see the whole picture.
Has solved freshness, dedup, and incremental indexing problems. You know that building the initial index is the easy part. Keeping it accurate, fresh, and deduplicated at scale is where the real engineering lives. You've built systems that handle continuous updates without full rebuilds and you've debugged the subtle correctness issues that come with incremental processing.
Self-directed experimenter who ships without handholding. You generate your own hypotheses, design your own experiments, and ship your own code. You don't wait for a roadmap or a sprint planning meeting. You see what needs to improve, you try something, you measure it, and you ship it if it works.
Backgrounds that tend to do well: Search engineers at companies with large-scale indexes (web search, e-commerce search, document search). IR researchers who've shipped their work to production. Infrastructure engineers who've built and operated real-time indexing pipelines. Engineers from Elasticsearch, Algolia, Vespa, or similar search infrastructure teams.

Responsibilities

Build and operate search indexes at massive scale: Design, build, and maintain the indexing infrastructure that powers Firecrawl's core product. You'll handle billions of documents and care about every millisecond of latency and every byte of storage.
Own the full stack from ingestion to serving: You don't just build one piece — you own the entire pipeline. Ingestion, processing, indexing, ranking, query understanding, and serving. When something breaks at 3am, you know where to look because you built it.
Solve ranking, relevance, and query understanding: Make sure the right content surfaces for the right queries. You'll build and iterate on ranking models, relevance scoring, and query parsing systems that directly impact product quality.
Tackle freshness, dedup, and incremental indexing: The web changes constantly. You'll build systems that keep our index fresh without re-crawling everything, deduplicate content intelligently, and handle incremental updates at scale without rebuilding from scratch.
Run experiments and ship results to production: You design experiments, measure results rigorously, and ship winners to production fast. You don't need someone to tell you what to try next — you have a backlog of ideas and the judgment to prioritize them.
Collaborate with the research team: Work closely with the Head of Research and the RL-focused Research Engineer to connect search/IR improvements with model training and broader product strategy.

Benefits

Salary that makes sense — $180,000–$270,000/year, based on impact, not tenure
Own a piece — Up to 0.15% equity in what you're helping build
Generous PTO — 15 days mandatory, anything after 24 days, just ask (holidays excluded); take the time you need to recharge
Parental leave — 12 weeks fully paid, for moms and dads
Wellness stipend — $100/month for the gym, therapy, massages, or whatever keeps you human
Learning & Development — Expense up to $1,000/year toward anything that helps you grow professionally
Team offsites — A change of scenery, minus the trust falls
Sabbatical — 3 paid months off after 4 years, do something fun and new
Full coverage, no red tape — Medical, dental, and vision (100% for employees, 50% for spouse/kids) — no weird loopholes, just care that works
Life & Disability insurance — Employer-paid short-term disability, long-term disability, and life insurance — coverage for life's curveballs
Supplemental options — Optional accident, critical illness, hospital indemnity, and voluntary life insurance for extra peace of mind
Doctegrity telehealth — Talk to a doctor from your couch
401(k) plan — Retirement might be a ways off, but future-you will thank you
Pre-tax benefits — Access to FSAs and commuter benefits (US-only) to help your wallet out a bit
Pet insurance — Because fur babies are family too
SF HQ perks — Snacks, drinks, team lunches, intense ping pong, and peak startup energy
E-Bike transportation — A loaner electric bike to get you around the city, on us