Member of Technical Staff, Retrieval & Indexing

Parallel Web Systems•Bay Area, CA

13d•$150,000 - $300,000•Onsite

About The Position

You will build and operate the systems that let Parallel run its own large-scale full-text index over the web. You will work across the full lifecycle: deciding what belongs in the corpus, building the searchable structures that hold it, building the signals and models that judge source quality and freshness, and serving queries against the index quickly and economically. You will anticipate bottlenecks before they appear and ensure the architecture evolves to meet increasing demand.

Requirements

Deep intuition on information retrieval.
Deep intuition on distributed systems.
Deep intuition on performance tuning.
Ability to reason about trade-offs between classical and neural retrieval.
Ability to reason about trade-offs between cost, reliability, and latency.
Care about helping agents find the right information.
Care about infrastructure that can support products used by millions without breaking a sweat.

Responsibilities

Build and operate systems for large-scale full-text indexing over the web.
Manage the full lifecycle of the index: corpus selection, structure building, quality/freshness modeling, and query serving.
Anticipate and resolve architectural bottlenecks.
Ensure architecture evolves to meet increasing demand.