Data Annotation Specialist, Software Engineering
About The Position
This role will focus on evaluating coding tasks, requiring you to review and debug code, navigate repository architecture, and analyze model trajectories. Your work will contribute to our model development efforts and the logic our models apply when completing task requests. This is a part-time independent contractor position available within Canada. We seek candidates who are able to commit to 16 hours per week minimum at a 40 CAD/hour contract rate. This role is BYOD - Bring Your Own Device (laptop). Remote work within Canada. 12 month contract. Performance incentives included!
Requirements
- Subject matter expertise in software engineering - you have both studied in this field and have 3-5 years of relevant industry experience.
- Proficient knowledge and understanding of Python and the following programming languages: Java, JavaScript, Go, and SQL (any dialect).
- Prior experience designing, building, and deploying APIs
Nice To Haves
- Prior experience working with code agents (OpenCode, Claude Code, Codex, Cursor) or evaluating agent trajectories is a big plus.
Responsibilities
- Evaluate the model's ability to respond to coding requests, workflows, and code base-related questions using available tools.
- Assess agent trajectories and model capabilities for code generation and debugging requests.
- Prompt models to complete complex coding tasks and review the accuracy of generated responses.
- Label, proofread, and improve machine-written and human-written software engineering-related outputs.
- Report quality and performance trends related to model/agent behaviour and project assignments.
Benefits
- Performance incentives included!
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Part-time
Career Level
Mid Level
Education Level
Associate degree
