Software Engineer 2

eBay•New York City, NY

22h•Hybrid

About The Position

eBay, Inc. seeks Software Engineer 2 in New York, NY. This role focuses on improving the effectiveness and efficiency of the “Similar Items” recommendation module on eBay’s product pages, which directly impacts millions of daily users. The engineer will build and maintain large-scale data processing pipelines using Scala and Apache Spark, aggregate user interaction signals, and utilize Hadoop Distributed File System (HDFS) to store and manage terabytes of behavioral and product data. Data will be preprocessed for quality and suitability for machine learning tasks. The role involves analyzing structured and unstructured data using SQL and Python (with libraries such as Pandas and NumPy) to extract user preferences and behavioral patterns. New features will be developed and integrated into the recommendation model, and deep learning-based recommended models using PyTorch will be implemented. Model performance will be assessed through offline evaluations and online A/B testing. Collaboration with product managers, data scientists, and backend engineers is essential to align recommendation strategies with user experience goals and business outcomes. Partial telecommuting is permitted from within a commutable distance.

Requirements

Master’s degree, or foreign equivalent, in Computer Science, Engineering (any field), or a closely related field.
Experience writing complex SQL queries to extract, join, and aggregate user behavior and item metadata from large-scale relational databases to train and evaluate recommendation models.
Experience developing and maintaining data pipelines, experimentation frameworks, and model training scripts using Python and related libraries such as Pandas, NumPy, and Scikit-learn.
Experience designing and training machine learning models for ranking, personalization, and user-item relevance prediction in large-scale recommendation systems.
Experience building and maintaining RESTful APIs to serve model predictions and integrate recommendation results into company’s buyer experience and experimentation platforms.
Experience implementing and optimizing algorithms for candidate retrieval and ranking to improve recommendation accuracy and diversity.
Experience utilizing efficient data structures (e.g., hash maps, heaps, trees, sparse matrices) to handle large-scale user-item interaction data and accelerate model computation.
Experience developing end-to-end recommendation pipelines including candidate generation, ranking, and post-processing to enhance personalization and engagement across company’s marketplace.
Experience leveraging large language models for semantic understanding of item titles and descriptions, query expansion, and improving cold-start recommendations.
Experience applying NLP techniques such as embeddings, text classification, and semantic similarity modeling to extract insights from listing text and user queries.
Experience building and validating predictive models to estimate user engagement metrics and forecast demand patterns for personalized recommendation strategies.
Experience using Hadoop and Spark to process terabytes of behavioral and transactional data, performing ETL to prepare features and datasets for large-scale machine learning training and evaluation.
Must be legally authorized to work in the U.S. without sponsorship.

Responsibilities

Improve the effectiveness and efficiency of the “Similar Items” recommendation module on eBay’s product pages.
Build and maintain large-scale data processing pipelines using Scala and Apache Spark.
Aggregate user interaction signals such as clicks, purchases, watchlist activity, and reviews.
Utilize Hadoop Distributed File System (HDFS) to store and manage terabytes of behavioral and product data.
Preprocess data to ensure quality, completeness, and suitability for downstream machine learning tasks.
Analyze structured and unstructured data using SQL and Python (with libraries such as Pandas and NumPy) to extract meaningful user preferences and behavioral patterns.
Develop and integrate new features into the recommendation model based on behavioral insights.
Implement deep learning-based recommended models using PyTorch, incorporating user and item embeddings, attention mechanisms, and custom loss functions.
Conduct both offline evaluations and online A/B testing to assess model performance.
Collaborate closely with product managers, data scientists, and backend engineers to align recommendation strategies with user experience goals and business outcomes.