Applied AI Engineer

Level•Austin, TX

11d

About The Position

The Applied AI Engineer designs, builds, deploys, and operates production agentic systems and LLM-powered features. The role owns the full delivery lifecycle, including prompts, tools, retrieval, guardrails, observability, cost and latency budgets, evaluation, and rollout. It also provides governance and security guidance for AI designs, strategy, and use. The engineer partners with subject-matter experts to identify high-value operational workflows, define success criteria, validate outputs, and automate meaningful portions of those workflows with systems the experts trust. SMEs participate in quality ownership from the beginning. AI features are treated as regulated products, with applicable compliance frameworks, safety and bias evaluations, human-in-the-loop processes, and content-filtering controls built into development and operation. Production handoffs include documentation, runbooks, evaluation harnesses, and dashboards. The role also helps teams adopt AI safely by enabling cross-functional teams to build and improve their own workflows and automations.

Requirements

At least 7 years of professional engineering experience.
At least 1 year of hands-on experience building agentic systems using a modern stack, such as LangGraph, Anthropic SDK or Claude Agent SDK, OpenAI Agents SDK, Pydantic-AI, Mastra, LlamaIndex, CrewAI, or a homegrown stack.
Experience building and operating agentic systems in production.
Strong familiarity with Python and one typed language for production services, such as TypeScript or Go.
Cloud experience with AWS or GCP and experience with containerized deployment.
A strong software engineering foundation and several years of production-environment experience, with a track record of leading systems to launch.
Experience shipping multiple LLM-powered features in production and the ability to discuss failures, fixes, and lessons learned.
Practical knowledge of agentic patterns including ReAct, structured-schema tool use, prompt chaining, routing, orchestrator-workers, evaluator-optimizer or reflection, and human-in-the-loop.
Ability to determine when a deterministic workflow is preferable to an autonomous agent.
Hands-on production experience with retrieval, including chunking, embeddings, hybrid search, re-ranking, metadata filtering, and their failure modes.
Working knowledge of grounding techniques, including citation and quote extraction, faithfulness and refusal evaluations, and post-hoc consistency checks.
Strong prompt-engineering practice, including zero-shot, few-shot, and many-shot patterns, example selection and ordering, in-context learning, and chain-of-thought.
Experience with structured output and production validation, using provider-native structured outputs, Instructor, Pydantic-AI, Outlines, or a comparable approach.
Disciplined evaluation practice and an ability to assess readiness beyond subjective review.
Experience standing up local models and creating and testing closed local model solutions.
Strong written and verbal communication skills, including the ability to explain architectural trade-offs to executives and junior engineers.
Comfort using AI coding tools extensively while owning problem framing, design choices, and verification.

Nice To Haves

Advanced retrieval experience, including GraphRAG, agentic retrieval, evaluation-driven retrieval tuning, or hybrid retrieval at scale.
Direct or transferable experience with safety, privacy, and policy constraints in user-facing AI; K-12 or other regulated-domain experience is a strong plus.
Experience with prompt-optimization frameworks such as DSPy, TEXTGRAD, or AdalFlow that have delivered production value.
A public repository, package, gist, technical write-up, or representative project demonstrating meaningful AI work.
Open-source contributions to AI tooling, including frameworks, agents, evaluations, or MCP servers.

Responsibilities

Design, build, deploy, and operate production agents and agentic workflows that solve defined problems end to end.
Own prompts, tools, retrieval, guardrails, observability, cost and latency budgets, and rollout for delivered systems.
Provide governance and security guidance for AI designs, strategy, and usage.
Identify high-leverage operational workflows and partner with subject-matter experts to define success criteria, validate outputs, and automate meaningful portions of their work.
Involve subject-matter experts in quality ownership from the start of projects.
Design AI features with relevant compliance frameworks in mind, such as COPPA, FERPA, and NIST CSF 2.0.
Run safety and bias evaluations before launch and continuously after launch.
Build human-in-the-loop and content-filtering controls for AI features before they reach end users.
Hand off production features with documentation, runbooks, an evaluation harness, and dashboards.
Foster safe AI adoption by enabling cross-functional teams to create and improve workflows and automations.