Sr. Research Scientist - Agent Flywheel

Databricks•New York, NY
•$164,000 - $235,000

About The Position

As a Research Scientist on Databricks’ Agent Flywheel team, you will build the context layer for recursive self-improvement and autofix loops for agents. This involves turning raw production session traces from Databricks data agents into online signals that agents can optimize against. You will research ways to convert real customer interactions into deterministic metrics, calibrated LLM judges, and failure clusters, and connect those signals to training and agent improvement. You will work collaboratively with researchers from diverse backgrounds and encode research findings into products customers rely on. The work spans evaluation and measurement for agents, including feedback signals, post-training datasets, benchmarks, and reinforcement-learning environments based on real customer workloads.

Requirements

  • Expertise in debugging and diagnosing failures in complex, stochastic, multi-component systems.
  • Strong instinct for designing metrics and evaluations that track meaningful outcomes, and for questioning noisy, sparse, or biased signals.
  • Experimental rigor and causal thinking, from offline ablations to online experiments.
  • Ability and motivation to turn measurement into improvements by connecting signals to changes that move the numbers.
  • Solid AI research and software engineering skills.
  • Strong communication skills and enthusiasm for working on a small, fast-paced team.
  • Drive to get work into the hands of real users.

Nice To Haves

  • Experience with agent systems, such as coding agents, long-horizon tool-using agents, or building evaluation environments.
  • A track record of novel research on learning from feedback, such as post-training, RLHF, or reward modeling.

Responsibilities

  • Turn production session traces from Genie Code and Genie One into online signals agents can optimize against.
  • Train and calibrate LLM judges so model feedback becomes reliable and repeatable, including calibration against expert human ratings.
  • Convert multi-turn agent trajectories into datasets and reward signals for post-training, connecting production signals, evaluation, and agent improvement.
  • Design benchmarks and reinforcement-learning environments based on real customer workloads to probe long-horizon, tool-using agent behavior.
  • Advance evaluation research, including online evaluations on live traffic, automated raters, verifiable rewards, and measurement for agents that act.
  • Keep up with developments in deep learning and create techniques that go beyond the state of the art.

Benefits

  • The total compensation package may include eligibility for an annual performance bonus and equity.
  • Comprehensive benefits and perks; specific details depend on region.
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service