About The Position
We are seeking an AI Test Scenario Designer to create realistic and structured evaluation scenarios for LLM-based agents. This role involves developing test cases that simulate human-performed tasks and defining ideal agent behavior for comparison. The goal is to ensure scenarios are clearly defined, easily scored, and reusable. The ideal candidate will possess a strong analytical mindset, meticulous attention to detail, and a keen interest in AI decision-making processes.
Requirements
- Currently pursuing or holding a Bachelor's or Master's degree in Computer Science, Software Engineering, Data Science / Data Analytics, Artificial Intelligence / Machine Learning, Computational Linguistics / Natural Language Processing (NLP), Information Systems, or related fields.
- Advanced (C1) or above English proficiency.
- Willingness to learn new methods.
- Ability to switch between tasks and topics quickly.
- Ability to work with challenging, complex guidelines.
- Possession of a laptop, internet connection, and available time.
- Enthusiasm to take on a challenge.
Responsibilities
- Designing structured test scenarios based on real-world tasks
- Defining the golden path and acceptable agent behavior
- Annotating task steps, expected outputs, and edge cases
- Working with developers to test scenarios and improve clarity
- Reviewing agent outputs and adapting tests accordingly
Benefits
- Paid up to $30/hour depending on skills, experience, and project needs.
- Part-time, remote, freelance project.
- Opportunity to work on advanced AI projects.
- Gain valuable experience that enhances your portfolio.
- Influence how future AI models understand and communicate.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
Upload and Match Resume
What This Job Offers
Job Type
Part-time
Career Level
Entry Level
