Software Engineer 2

eBay•New York, NY

2d•$131,997 - $229,600•Hybrid

About The Position

eBay, Inc. seeks Software Engineer 2 in New York, NY. This role will improve the effectiveness and efficiency of the “Similar Items” recommendation module on eBay’s product pages, which directly impacts millions of daily users. The engineer will build and maintain large-scale data processing pipelines using Scala and Apache Spark, aggregating user interaction signals. They will utilize Hadoop Distributed File System (HDFS) to store and manage terabytes of behavioral and product data, preprocessing it for quality and suitability for machine learning tasks. Analysis of structured and unstructured data using SQL and Python (with libraries such as Pandas and NumPy) will be performed to extract user preferences and behavioral patterns. New features will be developed and integrated into the recommendation model based on behavioral insights, leveraging domain knowledge and statistical methods. Deep learning-based recommended models using PyTorch will be implemented, incorporating user and item embeddings, attention mechanisms, and custom loss functions. Both offline evaluations and online A/B testing will be conducted to assess model performance, with results used to iterate and refine models. Collaboration with product managers, data scientists, and backend engineers is expected to align recommendation strategies with user experience goals and business outcomes. Partial telecommuting is permitted from within a commutable distance.

Requirements

Master’s degree, or foreign equivalent, in Computer Science, Engineering (any field), or a closely related field.
Experience writing complex SQL queries to extract, join, and aggregate user behavior and item metadata from large-scale relational databases to train and evaluate recommendation models.
Experience developing and maintaining data pipelines, experimentation frameworks, and model training scripts using Python and related libraries such as Pandas, NumPy, and Scikit-learn.
Experience designing and training machine learning models for ranking, personalization, and user-item relevance prediction in large-scale recommendation systems.
Experience building and maintaining RESTful APIs to serve model predictions and integrate recommendation results into company’s buyer experience and experimentation platforms.
Experience implementing and optimizing algorithms for candidate retrieval and ranking to improve recommendation accuracy and diversity.
Experience utilizing efficient data structures (e.g., hash maps, heaps, trees, sparse matrices) to handle large-scale user-item interaction data and accelerate model computation.
Experience developing end-to-end recommendation pipelines including candidate generation, ranking, and post-processing to enhance personalization and engagement across company’s marketplace.
Experience leveraging large language models for semantic understanding of item titles and descriptions, query expansion, and improving cold-start recommendations.
Experience applying NLP techniques such as embeddings, text classification, and semantic similarity modeling to extract insights from listing text and user queries.
Experience building and validating predictive models to estimate user engagement metrics and forecast demand patterns for personalized recommendation strategies.
Experience using Hadoop and Spark to process terabytes of behavioral and transactional data, performing ETL to prepare features and datasets for large-scale machine learning training and evaluation.

Responsibilities

Improve the effectiveness and efficiency of the “Similar Items” recommendation module on eBay’s product pages.
Build and maintain large-scale data processing pipelines using Scala and Apache Spark.
Aggregate user interaction signals such as clicks, purchases, watchlist activity, and reviews.
Utilize Hadoop Distributed File System (HDFS) to store and manage terabytes of behavioral and product data.
Preprocess data to ensure quality, completeness, and suitability for downstream machine learning tasks.
Analyze structured and unstructured data using SQL and Python (with libraries such as Pandas and NumPy) to extract meaningful user preferences and behavioral patterns.
Develop and integrate new features into the recommendation model based on behavioral insights, leveraging domain knowledge and statistical methods.
Implement deep learning-based recommended models using PyTorch, incorporating user and item embeddings, attention mechanisms, and custom loss functions.
Conduct both offline evaluations and online A/B testing to assess model performance.
Use results to iterate and refine models based on statistical significance and business KPIs.
Collaborate closely with product managers, data scientists, and backend engineers to align recommendation strategies with user experience goals and business outcomes.