Sr. Research Scientist - Agent Flywheel

Databricks•New York City, NY

About The Position

As a Research Scientist on the Agent Flywheel team, you will build the context layer for recursive self-improvement and autofix loops for agents, turning raw production session traces from Databricks’ data agents into online signals agents can optimize against. You will research how to convert real customer interactions into deterministic metrics, calibrated LLM judges, and failure clusters, then close the loop so those signals drive training and agent improvement. You will join a collaborative team of researchers and encode findings into products customers rely on. The role focuses on production session traces from Genie Code and Genie One, reliable model feedback, datasets and reward signals for post-training, benchmarks and RL environments based on customer workloads, and evaluation research for agents that act. You will keep up with developments in deep learning and create techniques that go beyond the state of the art.

Requirements

  • Expertise in debugging and diagnosing failures in complex, stochastic, multi-component systems.
  • Strong instinct for designing metrics and evaluations that track what matters, and for questioning noisy, sparse, or biased signals.
  • Experimental rigor and causal thinking, from offline ablations to online experiments.
  • Drive to turn measurement into improvement by closing the loop from signals to changes that improve results.
  • Solid AI research and software engineering skills.
  • Strong communication skills and enthusiasm for working on a small, fast-paced team.
  • Drive to get work into the hands of real users.

Nice To Haves

  • Experience with agent systems, such as coding agents, long-horizon tool-using agents, or building evaluation environments.
  • A track record of novel research on learning from feedback, such as post-training, RLHF, or reward modeling.

Responsibilities

  • Turn raw production session traces from Genie Code and Genie One into online signals agents can optimize against.
  • Train and calibrate LLM judges, including calibration against expert human ratings, so model feedback becomes reliable and repeatable.
  • Convert multi-turn agent trajectories into datasets and reward signals for post-training, connecting production signals, evaluation, and agent improvement.
  • Design benchmarks and reinforcement learning environments from real customer workloads to probe long-horizon, tool-using agent behavior.
  • Advance evaluation research, including online evaluations on live traffic, automated raters, verifiable rewards, and measurement for agents that act.
  • Keep up with developments in deep learning and create techniques that go beyond the state of the art.
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service