Applied AI/ML Engineer

Certum Legal Solutions

7h•$175,000 - $250,000•Hybrid

About The Position

Certum Legal Solutions (CLS) is a pioneering legal managed services organization (MSO) that supports mass-tort and single-event personal injury law firms. We provide the operational infrastructure, technology, and back-office solutions that allow the firms on our platform to grow and scale while focusing on delivering industry-leading results for their clients. We are building a proprietary, AI-native system that will transform every dimension of how CLS and its platform firms operate. This system encompasses AI-powered agents to manage end-to-end workflows throughout the lifecycle of mass-tort and single-event personal-injury litigation matters, including back-office support functions such as HR, accounting and finance, and IT. This is a greenfield build on a modern stack, designed from Day 1 to be extensible and, eventually, productized for the broader legal services market. We are assembling a high-caliber team to build something that does not exist in this form anywhere in the legal industry. If you want to see your work directly transform a real business and believe AI is fundamentally reshaping professional services, this is the team to join. ABOUT THE ROLE The Applied AI/ML Engineer owns the intelligence layer of Project Interstellar: the infrastructure that extracts structured data for robust document intelligence, powers conversational voice agents, orchestrates multi-step task agents, and handles various other operational workflows. This is not a research role. It is an applied AI/ML engineering role focused on building production LLM systems that perform reliably at scale in a HIPAA-regulated environment. You will work directly with the Lead Backend Engineer to integrate AI capabilities into the platform architecture from the ground up. WHAT YOU WILL BUILD • Production-grade structured data extraction pipeline for complex, multi-page domain-specific documents: entity extraction, classification, confidence scoring, temporal sequencing, and calculated output generation • Model abstraction layer with task-based routing, enabling cost-optimized inference across workloads of varying complexity (from high-volume classification to deep multi-step reasoning) • Agent orchestration framework and tool registry enabling task-executing agents to receive natural language instructions, plan multi-step executions, and call platform services • Quality assurance agent layer: automated rubric-based verification of upstream agent outputs against defined accuracy and completeness standards • Training and evaluation agent system: interactive scenario simulation and competency assessment for platform operators • RAG layer over domain-specific internal knowledge bases, enabling intelligent lookup and context injection for operational agents

Requirements

5+ years of software engineering experience with at least 2–3 years focused on production LLM application development
Expert Python; strong API integration skills; experience deploying AI systems at production scale
Deep familiarity with engineering reliable systems around Claude, GPT, or equivalent LLM APIs, not just calling them
Experience with agent frameworks (LangChain, LlamaIndex, or custom orchestration)
RAG architecture: vector databases, embedding models, retrieval optimization
Prompt engineering discipline: systematic evaluation, regression testing, version control for prompts
Experience handling sensitive or regulated data (HIPAA, FERPA, or similar)

Nice To Haves

Document and text extraction from unstructured sources (OCR post-processing, information extraction from clinical or legal documents)
Conversational AI or voice AI systems (NLU, dialog management, conversation evaluation)
Legal or healthcare AI applications
Multi-agent orchestration patterns
Experience evaluating and scoring LLM outputs at scale

Responsibilities

Own all AI/ML system design and implementation on the platform
Design and maintain prompt engineering infrastructure: versioned prompts, A/B evaluation, and regression testing
Build and iterate on structured extraction pipelines for complex, regulated documents, from initial parsing through multi-step reasoning and output validation
Design the agent orchestration framework powering task agents, QA agents, and operational agents
Establish AI quality standards: confidence thresholds, human escalation triggers, output validation
Partner with the Lead Backend Engineer on model abstraction layer design and API integration
Partner with domain experts on extraction schema design and workflow logic