Sr. AI Engineer

Vertafore • Denver, CO

6d • $175,000 - $210,000

About The Position

As a Senior AI Engineer, you will design, build, and operate the AI systems behind Vertafore’s next generation of products for agencies, MGAs, and carriers. You will build production-grade agents that can reason through complex, multi-step insurance workflows and make those systems reliable, measurable, and safe in a highly regulated industry.

You will work in small pods with complete agency to build end-to-end products. This is a hands-on role: you will directly contribute to the implementation and launch of critical features and make the decisions needed to deliver them. The pods are end-to-end development teams, so fluidity across the stack is expected.

Vertafore is seeking an engineer with strong software and systems engineering skills, a solid understanding of machine learning, and a track record of shipping. The role uses current agentic coding tools to work faster and requires accountability for the quality, security, and correctness of delivered systems.

Requirements

• Bachelor’s degree in Computer Science, Engineering, Physics, Mathematics, or a related field, or equivalent practical experience.
• 7+ years of professional software engineering experience, including significant experience building backend services and systems in Python.
• 2+ years building and shipping LLM-powered applications or agentic systems to production.
• Hands-on production experience with LangGraph or comparable agent orchestration frameworks such as LangChain, LlamaIndex, or custom orchestration.
• 5+ years designing and building RESTful APIs.
• Expert-level Python skills, including idiomatic, typed, tested, production-quality code and async programming.
• Solid understanding of machine learning fundamentals, including model training and generalization, overfitting, bias and variance, embeddings, classification and ranking metrics, and statistical significance.
• Deep understanding of practical large language model behavior, including failure modes, non-determinism, context limits, and strategies for reliable outputs.
• Demonstrated experience designing and running evaluations for ML or LLM systems and using results to drive measurable quality improvements.
• Experience with cloud platforms, Docker, and CI/CD pipelines.
• A rigorous, experimental mindset: forming hypotheses, measuring outcomes, and making decisions based on data.
• Accountability for the quality, security, and correctness of shipped work.

Nice To Haves

• FastAPI or similar framework experience.
• Experience with LLM observability and evaluation tooling such as LangSmith, Langfuse, Braintrust, or similar.
• Experience with fine-tuning, embedding model selection, or classical ML pipelines.
• Experience in insurance, financial services, healthcare, or another regulated industry.
• Experience with Model Context Protocol (MCP) or building tool integrations for agents.
• Open-source contributions or public writing on AI engineering.
• AWS cloud experience.

Responsibilities

• Design, build, and deploy complex reasoning agents and multi-agent systems for production workloads using LangGraph and related Python orchestration frameworks.
• Architect agent workflows, including state management, tool use, memory, planning, human-in-the-loop checkpoints, error recovery, and fallback behavior.
• Design and build clean, well-documented, versioned APIs that expose AI capabilities to Vertafore products and external partners.
• Define and own the evaluation strategy for AI systems, including building evaluation datasets, writing programmatic, model-based, and human-review graders, running offline and online evaluations, and using results to make improvements.
• Set up regression testing for prompts, models, and agent behavior so changes ship with measurable confidence.
• Instrument AI systems for production observability, including tracing, latency, cost, token usage, failure modes, and quality metrics.
• Build retrieval and context pipelines, including RAG, document processing, and structured extraction, over insurance data such as policies, submissions, and compliance documents.
• Troubleshoot and resolve production issues across the AI stack, from model behavior to infrastructure.
• Contribute hands-on to the implementation and launch of critical features in end-to-end product teams.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior