Senior Applied Researcher

eBay•Toronto, ON

2d•Onsite

About The Position

The AI Systems Performance & Governance team at eBay AI, Research and Innovation is looking for a highly qualified Senior Applied Researcher to join our team in Toronto. In this role, you will work at the intersection of applied research and AI governance for Generative AI systems (LLMs, VLMs, and agentic systems). You will focus on the science of evaluation, monitoring, and safety for GenAI applications, while contributing to the development of AI sandbox environments, evaluation frameworks, and governance best practices. You will help ensure that GenAI and agentic systems are scientifically grounded, properly evaluated, safe, and production-ready across their full lifecycle.

Requirements

Master’s degree or PhD in Computer Science, Engineering, Mathematics, or a related field.
Proven experience in machine learning, with strong hands-on experience building at least one of the following: LLMs, VLMs, Conversational Search systems, Agentic Systems, or Content Moderation solutions.
Proficiency in Python and frameworks such as PyTorch, TensorFlow, Langfuse, LangGraph, or similar.
Solid understanding of machine learning algorithms, model architectures, training techniques, and building performant inference pipelines.
Experience with data preprocessing, feature engineering, model evaluation metrics, and large-scale data processing frameworks such as Spark.
Solid understanding of the ML lifecycle, including experimentation, validation, and post-deployment monitoring.
Excellent analytical and problem-solving skills, and the ability to work in a fast-paced, dynamic environment.
Demonstrated experience independently leading complex applied research initiatives from problem formulation through production adoption, with evidence of influencing technical direction, mentoring others, and establishing methodologies or standards used beyond an individual project.
Strong communication and collaboration skills, with the ability to explain complex technical concepts to non-technical collaborators, propose creative solutions, and support tracking and delivery within release plans.

Nice To Haves

Experience with model inference optimization techniques and libraries is a plus.
Publication record in top AI conferences or journals is a strong plus.
Experience with AI safety, LLM/VLM/agent guards, content moderation, and policy-driven GenAI evaluation is a strong plus.

Responsibilities

Drive the fundamental science behind evaluation, creating robust methodologies and metrics to accurately assess the performance, safety, and reliability of LLMs, VLMs, and agentic systems, including offline metrics, human evaluation, and online experimentation.
Define scientifically grounded metrics for complex GenAI systems, including RAG pipelines and multi-agent workflows (e.g., hallucination, grounding, robustness, agent reliability, safety).
Provide scientific leadership across complex and ambiguous GenAI initiatives by setting research direction, reviewing evaluation and safety methodologies, and driving alignment on technical standards across teams.
Build and evolve AI Sandbox environments for safe experimentation, benchmarking, and validation of GenAI and agentic systems.
Advance the science behind building content moderation and safety guardrails, developing novel approaches for agentic safety steering to ensure autonomous and generative systems operate safely and ethically.
Establish scientific best practices and anti-patterns for GenAI and agentic application development, covering evaluation, safety, and system design.
Partner with engineering and product teams to ensure evaluation, safety, and monitoring approaches are applied consistently in production systems.
Translate Responsible AI requirements into measurable, testable, and scalable evaluation and safety solutions.