Applied AI Engineer — Conversational Agents (Senior/Staff)

Revin•New York, NY

11d•$225,000 - $325,000•Hybrid

About The Position

Revin is hiring a Senior- or Staff-level Applied AI Engineer for its Agents Platform team in Flatiron, New York City. The engineer will improve conversational AI agents through rigorous experimentation, using real customer conversations to identify shortcomings and ship production-ready improvements. The work spans instruction following, tool use, task completion, latency, and conversational experience, with a substantial focus on voice. The role owns the full improvement cycle: identifying problems, forming hypotheses, designing experiments, implementing changes, and validating impact. It involves improving agents through model selection, prompting, context, orchestration, and real-time voice behavior, while balancing quality, latency, reliability, and cost. At Staff level, the engineer will also shape evaluation and experimentation strategy across agent capabilities, identify high-impact opportunities, and help the broader team diagnose and improve agent behavior while remaining hands-on. The position is full-time and based in Revin’s Flatiron office. The team works in the office four days per week, with one remote day and flexibility when life comes up. Compensation is $225,000–$325,000 base salary, depending on level and demonstrated experience, scope of ownership, and impact, plus meaningful early equity. Benefits include health coverage, 401(k) matching, unlimited PTO, commuter benefits, office meals, a gym stipend, and technology equipment of the employee’s choice.

Requirements

Demonstrated track record improving substantial AI agent systems deployed to external customers, including the ability to explain system scale and complexity, identified problems, hypotheses and experiments, personal implementation, and evidence of improved production outcomes.
Strong experimental and analytical judgment, including defining useful baselines, designing informative comparisons, reasoning about noisy or confounded results, and learning from experiments that do not support a hypothesis.
Strong software engineering ability, including analyzing conversational data with code and SQL, debugging complex agent behavior, and turning experimental improvements into maintainable production software.
Practical understanding of agent systems and how models, instructions, context, tools, and orchestration interact, including the ability to weigh quality against latency, reliability, and cost.
Ownership, clear communication, ability to make progress through ambiguity, communicate findings and tradeoffs, and work closely with engineering and product teams.
Interest in early-stage startup work, broad responsibility, rapid iteration, and high standards; ability to use AI programming tools effectively while owning work quality.

Nice To Haves

Hands-on experience improving production voice agents, including diagnosing conversational timing, speech recognition and generation, audio quality, and end-to-end latency.
Experience building simulation, replay, or evaluation tools for multi-turn agents.
Experience developing human review workflows or calibrating automated evaluations against human judgment.
Experience at a strong startup or on a small team with substantial ownership.

Responsibilities

Analyze production conversations, listen to calls, inspect traces, and collaborate with customer-facing teams to identify failure patterns and prioritize improvements.
Define quality dimensions and representative evaluation datasets; combine human judgment, automated graders, and production metrics, and validate that measurements reflect customer experience.
Design reproducible experiments, compare approaches, and use offline evaluations and controlled production rollouts to understand results and tradeoffs.
Investigate voice-agent turn-taking, interruptions, response latency, transcription errors, speech quality, and spoken naturalness using audio and system telemetry.
Implement agent-system improvements, monitor their production effects, and take responsibility for reliability, cost, and regressions alongside quality gains.
Convert production failures and experimental findings into reusable datasets, evaluations, debugging tools, and release checks.
At Staff level, shape evaluation and experimentation strategy, identify high-impact opportunities, and raise the team’s ability to diagnose and improve agent behavior while remaining hands-on.

Benefits

Meaningful early equity.
Health coverage, including $0 employee premiums for individual medical coverage on most plans.
Dental insurance.
Vision insurance.
401(k) matching up to 4% of salary; contribute 5% to receive the full match.
Unlimited PTO.
Commuter benefits.
Free lunches and dinners in the office.
$100/month gym membership stipend.
Technology equipment of the employee’s choice.