Remote | Software & Data Science Benchmark Consultant — $50–$70/hour
About The Position
We are sharing a specialised part-time consulting opportunity for software engineering and data science professionals experienced in technical documentation, codebases, API references, data workflows, instruction-following evaluation, task authoring, and rubric design. This role supports current and upcoming remote consulting opportunities focused on professional document understanding, technical task design, workspace-file reasoning, code execution workflows, structured evaluation, and high-quality project execution. Selected professionals will author complex, multi-step tasks grounded in real-world technical materials, define accurate ground truth outputs, and create objective rubrics for evaluating structured responses.
Requirements
- 3+ years of hands-on experience in software engineering, data science, analytics, or a closely related technical field
- Strong ability to understand technical specifications, architecture documents, API documentation, codebases, or data workflows
- Experience writing clear technical instructions, reviewing structured outputs, or designing technical evaluation materials
- Comfort with code execution, debugging, data analysis, and technical reasoning
- Strong written communication skills and ability to define precise evaluation criteria
- High attention to detail and ability to create tasks with clear expected answers
- Ability to work independently in a remote, project-based environment
- Background in computer science, software engineering, data science, statistics, mathematics, analytics, information systems, or a related technical field may be highly relevant
- Practical experience in software development, data analysis, technical documentation, code review, QA, analytics engineering, or technical project work may also be valuable
- Equivalent professional experience may be considered depending on project needs
Nice To Haves
- Experience with benchmark design, evaluation design, task writing, technical assessment creation, or rubric-based review
- Familiarity with Python, SQL, APIs, notebooks, code execution environments, data pipelines, or developer documentation
- Experience reviewing technical documents, architecture diagrams, product specs, or software requirements
- Background in technical writing, developer education, QA testing, code review, analytics workflows, or documentation quality review
Responsibilities
- Design complex, multi-step tasks grounded in real-world technology workspace files
- Create tasks based on technical specifications, architecture documents, API references, codebases, data files, or analytical workflows
- Develop prompts that test precise instruction following, technical reasoning, and structured output generation
- Incorporate realistic requirements involving technical documentation review, web research, or code execution where appropriate
- Define clear ground truth outputs for each task
- Build objective evaluation rubrics that assess correctness, completeness, reasoning quality, and formatting
- Identify expected edge cases, common failure modes, and criteria for high-quality responses
- Ensure each task has a clear, reproducible evaluation path
- Review task materials for ambiguity, technical accuracy, and instruction clarity
- Validate that tasks are realistic, challenging, and aligned with professional software or data science workflows
- Check that expected outputs are accurate, well-structured, and defensible
- Maintain consistency, attention to detail, and strong documentation quality across submitted work
Benefits
- Competitive hourly compensation
- Flexible scheduling
- Weekly payments
- Projects may be extended, shortened, or adjusted depending on scope and performance
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Part-time
Career Level
Mid Level
Education Level
No Education Listed
