Prompt Engineer

Take2 • New York, NY

1d

About The Position

Take2 AI automates phone screens with AI agents. Take2's AI Interviewers automate the end-to-end screening process - evaluating resumes, conducting phone screens, and scheduling next round interviews with qualified candidates. Our AI Interviewers help reduce overhead costs while boosting speed-to-hire and preventing mis-hires. For candidates, this guarantees they’re never left without a response and delivers a better experience, uncovering skills that go beyond the resume. We are a team of Stanford GSB alums, backed by SemperVirens & Reach Capital. Our advisory board consists of ex CHROs of F500 companies such as Visa, HP, Disney & Google. Want a sneak peek of what we are building? Check out this video! https://youtu.be/UorNA5uuQjU About The Role Take2 AI is hiring a Prompt Engineer / AI Agent Engineer to design, implement, test, and scale our AI Interviewers and AI-based evaluation systems. Our AI agents already conduct tens of thousands of structured candidate interviews each month. This role sits at the intersection of LLMs, agent design, evaluation frameworks, and production systems, with direct ownership over interviewer behavior, question flow, scoring accuracy, and reliability at scale. This is a hands-on, highly analytical role for someone who enjoys translating ambiguous requirements into precise agent behavior, building rigorous evaluation pipelines, and continuously improving AI systems in production.

Requirements

• 2+ years of experience working with LLMs, NLP systems, or AI agents in production
• Demonstrated experience designing and deploying prompts, agents, or AI-driven workflows that operate at scale
• Strong understanding of prompt engineering techniques, agent control, and failure modes
• Experience building or contributing to evaluation, testing, or QA frameworks for AI systems
• Comfort working cross-functionally with engineering, product, and operations to translate requirements into system behavior
• Strong analytical mindset with the ability to reason about accuracy, consistency, bias, and edge cases
• Bachelor’s degree in Computer Science, Engineering, Math, or a related technical field

Nice To Haves

• Familiarity with voice or conversational AI systems, including real-time or high-volume environments
• Strong programming skills in Python (APIs, data pipelines, testing frameworks)
• Hands-on experience with multiple LLMs (GPT, Claude, Gemini, LLaMA, Mistral, or fine-tuned models)
• Experience designing multi-step AI agents with structured flows and state management
• Experience operating AI systems in production and iterating based on real-world performance metrics

Responsibilities

• Prompt Engineering & QA Design, build, and refine prompts that drive AI interviewer behavior, question sequencing, and responses
• Ensure prompts enforce role-specific requirements, structured interview flows, and consistent behavior
• Identify failure modes, edge cases, and sources of inconsistency in both conversations and evaluations
• AI Interviewer & Evaluation Development Implement AI interviewer agents based on customer-specific role requirements, screening attributes, and interview questions
• Design and maintain AI-based evaluation and scoring systems aligned with defined rubrics and hiring criteria
• Ensure evaluations are accurate, consistent, and explainable at scale
• Testing & Continuous Improvement Build and own evaluation pipelines to measure conversation quality, scoring accuracy, and reliability
• Simulate real-world interview scenarios to validate interviewer behavior before and after changes
• Monitor production performance and iterate rapidly to improve outcomes across high-volume interviews
• Reliability & Scale Ensure AI interviewers perform reliably and efficiently in high-throughput environments
• Partner with engineering to balance quality, latency, and cost tradeoffs
• Contribute to guardrails around consistency, bias, and compliance as systems scale
• Cross-Functional Collaboration Work closely with engineering to integrate prompts and agent logic into production systems
• Partner with product and operations to translate customer requirements into precise AI behavior
• Help shape internal standards and best practices for prompt engineering, evaluation, and AI quality

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level