AI Engineer

Momentic•San Francisco, CA

1d•$175,000 - $250,000•Onsite

About The Position

Momentic is building an AI-native testing platform that helps engineering teams verify that their products work and catch bugs before release. The company’s platform is used by engineering teams at Notion, Webflow, Xero, and Whatnot, and runs more than 1 million tests daily while processing more than 1 billion tokens per hour at peak. The AI Engineer will join Momentic’s engineering team and work closely with the CTO to improve the models and agents behind the platform. This hands-on role focuses on fine-tuning models, building evaluations, and using production data to make agents more reliable and efficient. The engineer will own technical projects from problem definition through delivery, making practical tradeoffs with the engineering team and shipping improvements. Momentic has raised a $15 million Series A led by Standard Capital, with backing from Y Combinator, FCVC, Transpose Platform, and Dropbox Ventures. Its customer base has grown fourfold year to date. The company has a team of 17 and is based in person in San Francisco at 650 5th Street.

Requirements

Typically 3 or more years of relevant software engineering experience, including hands-on contributions to AI-powered capabilities used in production.
Hands-on experience with supervised fine-tuning, including preparing training data and evaluating model behavior.
Experience with prompt and context tuning, evaluation datasets, and LLM observability to diagnose failures and measure improvements.
Strong engineering fundamentals and the ability to code efficiently.
Ability to use AI agents to build software by breaking down work, tracking progress, checking results, and redirecting agents when they get stuck.
Comfort with autonomy and ambiguity, including turning unclear problems into plans, managing work independently, and following through to delivery.
Demonstrated depth of contributions and shipped work; experience may include owning an AI capability within a larger product.

Nice To Haves

Experience with reinforcement learning fine-tuning, RL environments, or LLM-as-judge evaluators.
Experience with long-running agents, recovery from failed steps, or coordinating multiple agents.
Experience publishing benchmarks or running rigorous experiments on AI systems.

Responsibilities

Fine-tune models for long-running agents, including agents that investigate whether test failures stem from infrastructure, recent code changes, or test authorship.
Build and curate training and evaluation datasets from production workflows, using failure patterns to identify opportunities for improvement.
Develop LLM evaluations and observability that connect offline results with production behavior and help verify improvements.
Tune prompts, context, and model behavior to improve task success, reliability, latency, token usage, and cost.
Design repeatable experiments and help publish benchmarks demonstrating the performance of core agents on testing and browser workflows.
Own technical projects from problem definition through delivery, working with the engineering team to make practical tradeoffs and ship improvements.