Staff Software Engineer (AI Evals)

G2i Inc.

3d • $100 - $200

About The Position

This contract role evaluates the quality of interactions with modern coding agents, including OpenAI Codex, Claude Code, and Cursor. It is not a traditional engineering role and does not involve writing production code. The focus is on assessing whether AI coding agents reason and communicate like strong engineers, including the usefulness of their preambles and explanations, the quality of their engineering judgment, and whether interactions feel appropriate and trustworthy to experienced developers. The position calls for subjective but rigorous evaluation and clear, opinionated feedback.

Requirements

• Staff- or Principal-level engineer, or equivalent experience.
• Strong background in at least one of Go, Python, Rust, TypeScript, or JavaScript.
• Hands-on experience using OpenAI Codex, Claude Code, and Cursor.
• Deep familiarity with modern AI-assisted development workflows.
• Able to evaluate code without needing to execute it or deeply review every line.
• Comfortable giving direct, opinionated feedback.
• High standards for what constitutes good engineering.

Nice To Haves

• Experience with Cursor or similar AI-first IDEs.
• Prior exposure to prompt design or evaluation workflows.
• Experience mentoring senior engineers or defining engineering standards.

Responsibilities

• Evaluate AI-generated coding interactions end to end.
• Judge whether outputs are useful, broadly correct, and aligned with how a strong engineer would think.
• Assess explanations and reasoning, not just code.
• Distinguish between different levels of response quality.
• Provide clear, opinionated feedback on what worked, what did not, and what felt off or misleading.
• Help define what excellent interactions with tools such as Cursor look like.

What This Job Offers

• Career Level: Senior
• Education Level: No Education Listed