Data Scientist

Athari•Lexington, MA

12h•$75 - $100•Hybrid

About The Position

We are seeking a Data Scientist to design, develop, and implement methods, processes, and systems for consolidating and analyzing diverse data sets, including structured and unstructured data. This role involves developing software programs, algorithms, dashboards, information tools, and queries to clean, model, integrate, and evaluate datasets, while staying current with new analytic methodologies and technologies. The Data Scientist will collaborate with functional business units to drive business solutions and direction.

Requirements

5+ years of hands-on experience with Apache Solr or Lucene in production environments
Strong expertise in traditional relevancy engineering including query parsing, field boosting, function queries, and relevance tuning
Proven experience conducting relevancy analysis using both automated metrics and manual evaluation techniques
Strong expertise in vector embeddings and their application to semantic search
Proven experience building hybrid search systems that combine keyword and vector-based approaches
Knowledge of search relevance metrics (NDCG, MRR, precision/recall)
Excellent problem-solving and analytical skills
Strong communication skills and ability to work in collaborative environments
Currently holds a Secret Clearance (OR a higher clearance)
Quantitative relevancy analysis and tuning 5 years
Vector embeddings semantic search 5 years
C/C++, Java, Python, Bash, SQL, Java Script / HTML / CSS, Matlab 5 years
Apache Solr and creating data pipelines for search products 5 years
Data Analysis 5 years
R, Python, SQL, and Machine Language Algorithms and Data Analysis. 5 years

Nice To Haves

Databases and Data Engineering for Big Data
Elasticsearch
Statistical Methods
Databases and Data Engineering for Big Data 0 years
Elasticsearch 0 years
Statistical Methods 0 years

Responsibilities

Designs, develops, and implements methods, processes, and systems to consolidate and analyze diverse data sets including structured and unstructured.
Develop software programs, algorithms, dashboards, information tools, and queries to clean, model, integrate and evaluate datasets. Keeps abreast of new analytic methodologies and technologies.
Collaborate with functional business units to drive business solutions and direction.
Design, implement, and maintain enterprise-scale search solutions using Apache Solr
Develop and optimize semantic search capabilities using vector embeddings and neural search models
Build custom indexers and indexing pipelines that support vector embeddings alongside traditional text fields
Implement and tune Approximate Nearest Neighbor (ANN) algorithms for efficient similarity search at scale
Design and optimize similarity functions (cosine, dot product, Euclidean) for various search use cases
Build hybrid search systems that combine traditional keyword-based search with vector-based semantic search
Perform traditional relevancy engineering including query analysis, field weighting, boosting strategies, and result tuning
Conduct relevancy analysis using quantitative metrics and qualitative evaluation methods
Monitor search performance metrics and implement continuous improvements
Work cross-functionally with product, engineering, and data teams to define search requirements