Strategic Project Lead – Software Engineering

Careerflow.ai • , CO

3d • Remote

About The Position

Careerflow Human Data Labs partners with AI companies to bring real-world professional expertise into their products. The Strategic Project Lead will operate production software-engineering data and evaluation programs for frontier AI models, owning work across coding data, RLHF or preference data, SWE benchmarks, model evaluations, agentic coding, code generation, and code repair. The role covers contributor calibration, throughput, quality, rework, cost, timelines, and delivery recovery.

The position requires strong technical judgment and the ability to read code and tests, inspect final artifacts, identify invalid work, and make release or rejection decisions with minimal training. The role works remotely with global technology companies and AI labs, with flexible engagement options based on project requirements and availability. The stated commitment is 8 hours per day, with at least 4 hours of overlap with PST.

Requirements

• Direct experience operating production software-engineering data programs for frontier AI models, preferably involving coding training, coding RLHF, SWE benchmarks, model evaluations, agentic coding, code generation, or code repair.
• Proven ownership of the full production system, including contributor sourcing and calibration, task throughput, quality, rework, cost management, and delivery recovery.
• Strong software-engineering judgment, including the ability to read code and tests, inspect final artifacts, identify invalid technical work, and make release or rejection decisions.
• Experience managing distributed software-engineering contributors, reviewers, and team leads while maintaining consistent quality and delivery performance.
• Ability to take over an active program quickly, diagnose operational problems independently, and communicate clear decisions and recovery plans to demanding customers.

Responsibilities

• Operate active software-engineering data and evaluation programs end to end.
• Manage contributor sourcing, calibration, onboarding, reviewer capacity, and performance.
• Track throughput, quality, rework, defects, cost, and delivery timelines.
• Review code, tests, task metadata, and final artifacts for technical correctness.
• Diagnose quality failures and improve task design, rubrics, instructions, and review processes.
• Lead recovery plans when programs fall behind on quality, throughput, cost, or timelines.
• Coordinate distributed software engineers, reviewers, and team leads.
• Communicate progress, risks, and corrective actions to customer research and program teams.
• Use Python, SQL, or similar tools for quality analysis, defect tracking, and operational reporting.
• Take over active programs, independently diagnose operational problems, and communicate decisions and recovery plans to customers.

Benefits

• Remote work with leading global technology companies and AI labs.
• Flexible engagement options based on project requirements and availability.
• Opportunity to work on high-impact AI training, evaluation, and data-generation programs.
• Collaboration with experienced technical, research, operations, and delivery teams.
• Opportunity to build expertise at the intersection of software engineering, AI, and large-scale operations.

What This Job Offers

• Career Level: Manager
• Education Level: No Education Listed