Member of Technical Staff: Agent DX Research
About The Position
Modal provides the infrastructure foundation for AI teams. With instant GPU access, sub-second container startups, and native storage, Modal makes it simple to train models, run batch jobs, and serve low-latency inference. We have thousands of customers who rely on us for production AI workloads, including Lovable, Scale AI, Substack, and Suno. We're a fast-growing team based out of NYC, SF, and Stockholm. We've hit 9-figure ARR and recently raised a Series B at a $1.1B valuation. Our investors include Lux Capital, Redpoint Ventures, Amplify Partners, and Elad Gil. Working at Modal means joining one of the fastest-growing AI infrastructure organizations at an early stage, with many opportunities to grow within the company. Our team includes creators of popular open-source projects (e.g. Seaborn, Luigi), academic researchers, international olympiad medalists, and experienced engineering and product leaders with decades of experience. Modal has always obsessed over developer experience and productivity. With rapid advancements in the capabilities of AI coding agents, the practice of developing software and the meaning of developer experience is changing. We see this as an opportunity. We’re looking for an experienced researcher to join us and help make it even easier and more productive to build on Modal. We believe that our code-first approach to AI infrastructure is uniquely well suited to agent-based development. But we’re looking to do even better by subjecting agent productivity to rigorous evaluation and using those insights to guide the development of our platform. You’ll work in collaboration with Modal’s SDK team and other product engineers to build out a framework and process for agent productivity evaluation. Our goal is to treat developer experience optimization as a scientific problem. You’ll be responsible for defining quantitative objectives, designing systems to measure performance, and translating results into product improvements. You’ll also be expected to stay on top of new developments in tools and workflows and to work with our customers to understand how they’re using coding agents with Modal and where we can be providing more value.
Requirements
- Sufficient technical skills to design and implement scalable agent benchmarking workflows
- Experience with experimental design, measurement, and statistical evaluation
- Up-to-date knowledge of the latest advances in coding agents (with a dose of healthy skepticism about their current capabilities)
- Interest in developer tooling and opinions about developer ergonomics
- Familiarity with the use cases that Modal serves (generative AI inference, large-scale batch jobs, multi-node training, etc.)
- Strong communication skills and the ability to convey research insights to decision makers
- The ability to work in person from our New York (preferred) or San Francisco office
Responsibilities
- Defining quantitative objectives
- Designing systems to measure performance
- Translating results into product improvements
- Staying on top of new developments in tools and workflows
- Work with our customers to understand how they’re using coding agents with Modal and where we can be providing more value.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Mid Level
Education Level
Ph.D. or professional degree
