Sr/Lead AI Harness Engineer

FICO

1d•$105,000 - $165,000

About The Position

FICO is seeking a Sr/Lead AI Harness Engineer to join their engineering team in a hands-on technical role focused on a new discipline: Harness Engineering. As AI coding agents become more prevalent, the challenge shifts from writing code to verifying and trusting AI-generated code. Harness Engineering aims to create an environment that guides AI agents toward correct, maintainable, and well-architected output, enforcing quality through the system. The Lead Harness Engineer will be responsible for building and owning the individual controls within this harness, managing the process from problem identification to production deployment. This role emphasizes systems and leverage over manual application code writing.

Requirements

Bachelor's/Master's in Computer Science or related disciplines, or relevant experience in software architecture, design, development, and testing.
Strong software engineering background with experience in large codebases, focusing on architecture, testing, and maintainability.
Proficiency in building tooling across a modern stack, including linters, static analysis, CI pipelines, containerized build/test environments, and instrumentation/observability.
Familiarity with agent instruction conventions such as AGENTS.md.
Hands-on experience with AI coding agents (e.g., Claude Code, Codex, or similar) and an understanding of their strengths and weaknesses.
Experience with spec-driven development, context engineering, agent orchestration, fitness functions, and developer-platform work.
A systems mindset focused on fixing the environment rather than individual outputs, with the ability to translate quality standards into mechanical, repeatable rules.
Judgement in choosing between deterministic controls (type checkers, linters) and inferential, LLM-based controls (AI code review, LLM-as-judge), understanding their trade-offs.
Clear communication skills for articulating designs with architects and discussing strategy with teams.

Responsibilities

Design, develop, deploy, and support components of the harness, including guides, feedback loops, guardrails, and shared context to ensure production-grade output from AI agents.
Build feedforward guides such as agent instruction files, reusable skills, architectural rules, reference documents, and codemods to improve initial AI output accuracy.
Develop feedback sensors like custom linters, static analysis tools, structural and architecture-fitness tests, verification loops, and LLM-as-judge reviewers to identify issues before human review.
Manage the steering loop to prevent repeated mistakes by agents and maintain repository knowledge (docs, specs, context) for agent accessibility, combating drift.
Contribute to quality gating, release criteria, and LLM testing to ensure AI-generated output meets quality and safety standards.
Enhance observability into agent work and track key metrics such as cost per merged PR, time-to-merge for agent-assisted PRs, review velocity relative to PR size, defect escape rate, and agent-PR survival rate.
Evaluate the stability, compatibility, scalability, interoperability, and performance of harness components.
Continuously learn new techniques in agent-augmented engineering and serve as a technical expert and mentor to junior team members.