Remote | Software Engineer — $90–$140/hour
About The Position
We are sharing a specialised part-time consulting opportunity for experienced Software Engineers to contribute technical expertise to an advanced AI development project focused on evaluating technical content, software-related reasoning, and AI-generated system outputs. This engagement focuses on critical analysis and evaluation rather than software development or feature delivery. Selected professionals will assess AI-generated responses to technical prompts, determine whether outputs are accurate, clearly communicated, appropriate for their intended audience, and genuinely useful, and provide detailed written reasoning to support each evaluation. No prior experience in AI is required.
Requirements
- Professional experience as a Software Engineer working with a modern technology stack
- Native-level English fluency and exceptional written communication skills
- Proven experience authoring or reviewing technical documentation, design documents, API references, specifications, or similar materials
- Experience conducting code reviews, design reviews, or providing structured feedback on technical writing
- Strong ability to assess technical communication for correctness, audience appropriateness, and instructional value
- Excellent analytical skills and meticulous attention to detail
- Ability to identify subtle factual inaccuracies, logical inconsistencies, ambiguous explanations, and misleading technical language
- Strong independent-working skills and comfort operating within remote, asynchronous project environments
Nice To Haves
- Familiarity with AI model evaluation, annotation, RLHF, or related structured evaluation workflows is advantageous but not required
Responsibilities
- Evaluate and compare AI-generated responses to technical prompts for correctness, clarity, tone, and usefulness
- Review outputs involving technical documentation, error messages, code comments, architectural explanations, instructions, and related software-engineering content
- Determine whether generated responses accurately address the underlying technical question or requirement
- Identify outputs that appear fluent or convincing but contain factual, logical, or technical inaccuracies
- Assess whether technically correct responses are communicated clearly and appropriately for their intended audience
- Review technical writing for precision, completeness, structure, and instructional value
- Evaluate documentation, design documents, API references, specifications, and comparable technical materials
- Identify unclear explanations, ambiguous wording, misleading statements, inconsistencies, and missing information
- Assess whether technical content is appropriately tailored to developers, technical stakeholders, or other intended users
- Apply professional software-engineering judgement when reviewing the quality of technical communication
- Assign analytical quality scores to AI-generated technical responses
- Write detailed rationales explaining the reasoning behind each evaluation
- Clearly distinguish between factual correctness, communication quality, relevance, and overall usefulness
- Document subtle errors, inconsistencies, or ambiguities with precision and consistency
- Provide actionable written feedback that can support improvements in AI model performance
- Work independently within structured remote evaluation workflows
- Collaborate asynchronously with project managers, reviewers, and other expert contributors
- Follow established evaluation guidelines, rubrics, and project-specific quality standards
- Apply consistent judgement across repeated technical assessment tasks
- Complete assigned work within agreed project requirements and defined timeframes
Benefits
- Part-time independent contractor engagement
- Fully remote
- Compensation: $90–$140/hour
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Part-time
Career Level
Senior
Education Level
No Education Listed
