AI Engineer

Novella • New York, NY

26d

About The Position

We're building Novella on the belief that AI should touch every part of how E&S insurance is designed, sold, and serviced. We need an AI Engineer who doesn't just build models, but builds products. You'll own the development of agentic workflows and LLM-powered systems that sit at the core of what we deliver to brokers and carriers every day.
This isn't a research role. You'll ship autonomous agents and LLM applications into production, design evaluation frameworks to make sure they actually work, and push the boundaries of what's possible with AI in an industry that barely understands technology. You'll also be our internal AI evangelist — the person who makes sure every team at Novella, technical or not, is leveraging the latest AI tools to move faster and do better work.

Requirements

• Proven track record as a software engineer with 3+ years of experience developing production systems.
• Deep understanding of LLM application development, including practical experience with:
• LLM evaluation and monitoring systems
• AI agent architectures and frameworks
• Retrieval-Augmented Generation (RAG)
• Strong background in dataset development and benchmark creation for LLM applications.
• Experience taking ownership of projects from conception to completion, with strong initiative and iterative development skills.
• Ability/Desire to work with TypeScript

Nice To Haves

• Hybrid skill set combining data driven mindset with engineering capabilities.
• Track record of successfully deploying and maintaining ML/LLM systems in production environments.
• Experience with performance optimization and scaling of LLM applications.
• Experience working on AI agents.

Responsibilities

• Lead the development of autonomous agents and LLM-based applications, covering the entire development lifecycle from initial design through production deployment and optimization.
• Design and develop comprehensive evaluation frameworks, including datasets and benchmarks, to ensure our LLM applications meet quality standards and perform reliably over time.
• Provide mentorship to team members, sharing your expertise in LLM development and helping build a strong AI first technical culture

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Education Level: No Education Listed