AI DevEx Engineer

Incident IQ•Atlanta, GA

2d•Hybrid

About The Position

We are looking for a pragmatic and forward-thinking AI DevEx Engineer to join our AI Tooling team. In this role, you will be instrumental in building the skills, plugins, commands, and internal tools that empower our engineering team and cross-functional departments to work smarter and faster. Initially, your primary focus will be expanding our custom plugins and agentic skills for Claude Code, which is heavily utilized by our engineering teams. As we grow, you will scale these efforts into multiple targeted plugins across different engineering disciplines. We need someone who is relentlessly focused on solving real business problems through Full Stack Agentic Development (FSAD). You will architect practical AI solutions, shape the "brain" of our agents, pioneer how we test AI outputs, and clearly articulate your decision-making process to the team. We are building complex, non-deterministic systems, which means this role requires deep resilience, patience, and a persistent focus on long-term goals despite inevitable setbacks.

Requirements

Strong command of LLM/Agentic orchestration, including MCP integrations, AI debugging/evaluation methodologies, and core CS fundamentals. A focus on data security and privacy is critical.
Exceptional ability to listen, translate complex technical jargon into actionable insights for non-technical stakeholders, and advocate for user-centric solutions.
A pragmatic approach to problem-solving that favors simple, scalable solutions over over-engineering. We value autonomy, adaptability to rapidly shifting AI landscapes, and a positive alignment to advancing our AI strategy.
Strong general understanding of coding and architectural understanding, with highly preferred experience in .NET (backend) and React (frontend).

Nice To Haves

Experience with Retrieval-Augmented Generation (RAG) pipelines, vector databases, or knowledge graphing.
Experience building internal Developer Experience (DevEx) tools.
Familiarity with AI evaluation (Evals) frameworks for scoring agent performance.

Responsibilities

Design and build out robust skills, tools, and agents for our internal Claude Code plugins, eventually scaling this architecture to support multiple distinct engineering workflows.
Manage, structure, and expand our internal reference library. You will be responsible for ensuring our agents have the right context and documentation to make highly accurate, business-aligned decisions.
Actively review agent runs, interactions, and logs generated by the engineering teams. You will identify failure points, debug complex agentic loops, and deploy fixes to improve reliability.
Design and implement a comprehensive testing suite from the ground up tailored for Full Stack Agentic Development. You will establish robust methodologies to effectively test, evaluate, and benchmark non-deterministic AI results.
Make sound architectural decisions across the stack (primarily utilizing .NET and React) to ensure scalability, security, and performance of our AI infrastructure.
Clearly document and explain your code, architectural choices, and the step-by-step process used to achieve your solutions to technical and non-technical stakeholders.
Actively mentor and teach engineering teams, creating comprehensive documentation and maintaining an open, feedback-receptive mindset toward the tools you build.
Approach development with deep empathy for existing workflows, ensuring that new solutions enhance our current processes rather than disrupting them for the sake of 'the latest' tech.