Staff Software Engineer, Search & Retrieval Infrastructure

Syllo

11d•$190,000 - $230,000•Remote

About The Position

Syllo is seeking a Staff Software Engineer to take ownership of their advanced search, indexing, and data scanning infrastructure as the company scales to handle multi-petabyte data volumes. The role involves optimizing and evolving the retrieval stack to ensure sub-second latency for interactive workflows while designing cost-effective architectures for deep scanning and vectorizing massive amounts of cold-storage data. The engineer will lead the design and implementation of data tiering and retrieval strategies to maintain peak performance and manage cloud compute costs.

Requirements

8+ years of software engineering experience, with a proven track record operating at the Staff/Principal level optimizing and scaling highly distributed, high-throughput systems to handle petabyte-level data.
Deep, production-level expertise tuning and scaling Lucene-based search engines (Elasticsearch, Solr) and modern vector indexing infrastructure. You deeply understand index internals, chunking strategies, and embedding retrieval optimization.
A strong history of managing the compute vs. storage trade-off. You know how to design sophisticated cold-storage scanning solutions and hot-index architectures that are highly performant but fundamentally cost-effective.
Extensive experience managing complex data pipelines, high-throughput event streaming (Kafka, Kinesis), and distributed compute architectures handling billions of records.
Expert command of cloud primitives (GCP preferred), Kubernetes, and infrastructure-as-code.
Expert-level proficiency in systems-level and backend languages (Go, Rust, Python, or Java/C++).

Responsibilities

Scale the Retrieval Stack: Lead the optimization and architectural evolution of our existing hybrid search infrastructure, maximizing the throughput and efficiency of both lexical search (e.g., Elasticsearch, Lucene) and dense vector databases.
Advanced Data Tiering & Scanning: Design and implement intelligent, cost-effective tiering strategies across hot, warm, and cold data states. Evolve our distributed pipelines to efficiently execute asynchronous, massive-scale scans of petabytes of data in varying states of availability.
Relentless Optimization: Drive down latency and cost-to-serve. Deeply analyze system bottlenecks, tune indexing and querying algorithms, and optimize cloud infrastructure (compute, storage, and networking) for maximum efficiency at extreme scale.
Technical Leadership: Act as the domain expert and owner of the indexing and search ecosystem. Set the long-term technical vision for data storage and retrieval, guiding engineering teams on best practices for high-volume data modeling and performance tuning.
Resiliency at Scale: Ensure fault-tolerant, highly available operations during massive parallel ingest events and complex, concurrent querying across millions of documents.