Applied AI Engineer

OpenLoop Health•Des Moines, IA

About The Position

OpenLoop is hiring for this role at the Senior, Staff and Senior Staff levels. The Applied AI Engineer will build and run LLM-based agents and assistants that real users depend on. This includes building the agent runtime and orchestration, evaluation sets, test harnesses, regression tests, and AI-graded evals. The role also involves working across multiple model providers, managing version upgrades, and planning fallbacks. Additionally, the engineer will focus on observability and cost, tracing agent behavior, monitoring production, and attributing AI spend. Grounding and retrieval will connect agents to company data, and workflow design will determine which human workflow steps an agent can reliably perform. Security, privacy, and safety are paramount, ensuring patient data (PHI) is protected and agents are controlled. The role will build on GCP and partner with Data Platform teams. Engineers are expected to contribute to code and design reviews, help newer engineers, and explain AI trade-offs to product, operations, and clinical stakeholders.

Requirements

Bachelor's degree in Computer Science, Engineering, or a related technical field
5 to 15+ years of software engineering experience.
Hands-on experience building and running LLM-based systems in production that real users relied on.
Real hands-on depth in at least one core area: agent runtime, evaluation, retrieval, or observability and cost.
Experience measuring whether an AI system works with evaluation data.
Strong production engineering across testing, debugging, services, APIs, deployment and on-call.
Experience handling sensitive data with a security-first mindset.
Comfort working in a greenfield space, and clear communication with technical and non-technical partners.

Nice To Haves

Experience in regulated industries like healthcare (PHI, HIPAA), finance, or insurance.
Experience running production services on Google Cloud (GCP).
Experience migrating a system between model versions or providers.
Machine learning experience beyond LLMs.
Built a platform used by other engineering teams.
For Senior Staff, experience leading a small team's technical direction from zero and growing a single pod into multiple teams.

Responsibilities

Build and run LLM-based agents and assistants that real users depend on.
Build agent runtime and orchestration, including failure handling, retries, timeouts, and sandboxing.
Build evaluation sets, test harnesses, regression tests, and AI-graded evals.
Work across multiple model providers, manage version upgrades, and plan fallbacks.
Trace agent behavior in production and attribute AI spend.
Connect agents to real company data through retrieval (RAG), context design, and prompt engineering.
Work with operations teams to decide which steps of a human workflow an agent can do reliably.
Protect patient data (PHI) in every AI system, control agent access, and defend against misuse.
Build on GCP and partner with Data Platform teams.
Contribute to code and design review, and help engineers newer to AI.
Explain AI trade-offs in plain language to product, operations, and clinical stakeholders.