Agent Architect

Pencil

4d • Remote

About The Position

At Pencil, we’re building the agentic OS for marketing. We aren't just using integrating generative AI; we are building the machine that makes it professional, brand-safe, and scalable. We’re moving beyond simple text-in/text-out interfaces toward complex multi-agent architectures that can handle the nuanced demands of global brands and the agility of small businesses. We are looking for an Agent Architect who thinks in systems, not just sentences. You will be responsible for the "brain" of Pencil—designing the agent logic, tool-calling structures, and evaluation loops that power our core creative engine and bespoke client solutions.

Requirements

• 3+ Years of Direct GenAI Experience: You have a deep, intuitive, and technical understanding of LLMs (GPT-4, Claude, Gemini) and multimodal models (Stable Diffusion, Midjourney, Video Gen).
• Systems Thinking: You don’t just write a prompt; you think about the latent space, the context window, and how one agent's output becomes another’s input.
• Technical Literacy: While this isn't a "Software Engineer" role, you should be comfortable with Python, JSON structures, and API documentation.
• Evaluation Obsession: You believe that if you can’t measure a prompt’s performance, you shouldn’t ship it. You are familiar with benchmarking and A/B testing AI outputs.
• The "Creative/Technical" Bridge: You can sit in a room with a Creative Director and translate "make it feel more punchy" into a temperature adjustment and a few-shot prompting strategy.

Nice To Haves

• Experience with orchestration frameworks (LangChain, CrewAI, AutoGen) is a major plus.

Responsibilities

• Design and implement multi-agent workflows, including task decomposition, state management, and tool-use (RAG, API integration, etc.).
• Implement rigorous evaluation frameworks (e.g., using LLM-as-a-judge, promptfoo, or DSPy) to measure and improve agent performance at scale.
• Develop reusable agents and prompt libraries that allow the platform to serve 1,000+ brands with unique voices simultaneously.
• Partner with AI Engineering and Product teams to determine which behaviors should be handled via prompting, RAG, or fine-tuning.
• Act as the technical lead for complex client deployments, translating high-level creative briefs into deterministic AI workflows.

Benefits

• 25 days PTO plus public holidays, although we operate a Flexible Time Off scheme
• Health insurance / private medical cover
• Monthly stipend towards wellness, fitness, and learning and development
• Remote - work from anywhere in your home country
• Enhanced parental leave policies, whether you become a parent through birth, adoption or surrogacy
• Access to our Pencil office in The Shard, London for our UK employees
• Flexible working hours

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed