Agentic AI/LLM Engineer, Coverage Intelligence

Bold Penguin•Dublin, OH

4d•Remote

About The Position

Bold Penguin is hiring an Agentic AI/LLM Engineer for its Coverage Intelligence work. Bold Penguin is a digital broker for commercial insurance whose technology supports digital distribution, wholesale capabilities, and AI-driven insights across the commercial insurance lifecycle. In this role, you will design, build, and own production AI systems for document understanding and coverage analysis. You will develop agentic workflows combining models, tools, and human review, with outputs grounded in source documents and domain knowledge so results are traceable and explainable. You will improve accuracy through evaluation and error analysis, make trade-offs among model accuracy, cost, and latency, and work with domain experts and engineers to translate expert judgment into durable system behavior. You will also help guide the team’s technical direction as LLM and agent capabilities evolve. This is a fully remote role except for onboarding and optional in-office events. New-hire onboarding and orientation take place at the Bold Penguin office in Dublin, Ohio, for two days and two nights; travel costs are covered. Occasional travel of up to 10% may be required for corporate events and team planning sessions.

Requirements

At least 5 years of experience in software engineering or applied machine learning, including at least 2 years building LLM applications in production.
Strong Python skills, including experience with typed schemas for structured output such as Pydantic.
Hands-on experience extracting structured data from complex documents using PDF parsing, OCR, and vision-language models.
Experience with agentic patterns such as tool use, multi-step workflows, self-verification, and human-in-the-loop review.
Experience with retrieval-augmented generation, embeddings, and source-grounded output.
A rigorous approach to evaluation, including defining metrics, building test sets, and conducting systematic error analysis.
Excellent written and verbal communication skills, including the ability to work directly with domain experts.
Ability to work autonomously with minimal direction, navigate ambiguity, and maintain a high bar for quality.
Comfort with the pace and changing priorities of a startup, including building a function while operating it.
Ability to sit, stand, or walk for prolonged periods, up to 8 hours per day, while working at a computer.
Ability to use standard office equipment for extended periods, including a mouse, keyboard, phone, and video conferencing equipment.
Ability to travel to the Dublin, Ohio office for two days and two nights for new-hire onboarding and orientation.
Ability to travel occasionally, up to 10%, for corporate events and team planning sessions.

Nice To Haves

Experience with insurance, legal, financial, or other contract-heavy documents.
Experience with LangGraph or similar agent frameworks.
Experience with LLM observability and evaluation tooling.

Responsibilities

Design, build, and own production AI systems for document understanding and coverage analysis.
Build agentic workflows that combine models, tools, and human review, recognizing when a decision requires an expert.
Ground outputs in source documents and domain knowledge so results are traceable and explainable.
Drive accuracy through evaluation, experimentation, and error analysis against expert-labeled data.
Make trade-offs among models and providers based on accuracy, cost, and latency.
Partner with domain experts and engineers to turn expert judgment into durable system behavior.
Help set the team’s technical direction as LLM and agent capabilities evolve.
Perform other duties and responsibilities as assigned.