Search Engineer, AI Infrastructure (SF)

Lavendo•San Francisco, CA

4d•$235,000 - $260,000•Onsite

About The Position

Lavendo partners with startups and high‑growth companies to help them hire top‑tier sales, GTM, and technical talent. This role is with one of our clients; we’ll share full details about the company and interview process as we get to know you and confirm mutual fit. Our client builds the data layer that AI agents run on. One API call turns any URL into clean, structured, LLM-ready data. It is already the default way a huge number of developers pull web data into their AI systems. The numbers back it up: 8 figures in ARR in year one, more than doubled that in year two, and 125,000+ GitHub stars. They just closed a $75M Series B. The company is small, moving fast, and hiring people who want to build the next chapter with them. They believe developers and AI agents should not have to fight the web to get useful data out of it. Every hour spent maintaining a scraper, dealing with stale pages, or working around brittle retrieval systems is an hour taken away from the actual product. Their job is to make that problem disappear: give AI systems reliable access to the live web, then make the results clean, current, structured, and useful enough to power real products. As a Search Engineer, you will own the systems that turn the open web into a high-quality search index. You will work directly with the Head of Search and a small team to build and improve the crawling, indexing, retrieval, ranking, and serving infrastructure behind a fast-growing search product. This is an end-to-end role: you will improve relevance while solving the real constraints of search at scale - freshness, deduplication, latency, reliability, and cost per query. The key requirement is direct production ownership of ranking and relevance. You should be able to point to a search-quality change you personally shipped and the metric it improved.

Requirements

3+ years building and operating production search or retrieval systems, with direct ownership of ranking and relevance
A specific example of a ranking or reranking change you owned and the measurable impact it had on relevance, recall, conversion, CTR, NDCG, MRR, or a similar metric
Hands-on information-retrieval depth across BM25/lexical, semantic/vector, hybrid retrieval, reranking, and retrieval or RAG evaluation
Strong backend and distributed-systems skills in Go, Rust, Python, or a comparable language, with end-to-end production ownership
Experience improving latency and cost at meaningful query volume while maintaining quality and reliability
Familiarity with search and infrastructure tooling such as Lucene, Elasticsearch, OpenSearch, Redis, Kafka, PostgreSQL, Kubernetes, Docker, AWS/GCP, gRPC, or Protobuf

Nice To Haves

The people who do well here have personally owned search quality in production, think end-to-end about relevance, crawling, indexing, serving, latency, and cost, and thrive in ambiguity by shipping, measuring, and iterating quickly without needing a fully defined playbook.

Responsibilities

Own ranking quality and relevance for LLM-driven search and retrieval
Build and operate the crawling, indexing, retrieval, and search-serving systems behind the product.
Improve search quality using lexical, semantic, hybrid, and reranking approaches
Reduce latency and cost per query as query volume and corpus size grow, without compromising relevance, correctness, or reliability
Own systems end to end: design, build, deploy, monitor, troubleshoot, measure, and iterate in production
Solve for freshness, incremental indexing, deduplication, crawl scheduling, and large-scale corpus management
Work directly with the Head of Search and the team to shape the roadmap and ship customer-facing improvements quickly