Data Analyst
About The Position
Despite rapid development in generative AI, quality measurement remains challenging due to the lack of reliable and robust technical metrics. This is especially the case when the slight differences in the details generated by the model could have a significant impact on user experience. In this team, we are developing a series of manual (human rating), semi-automatic, and fully automatic evaluations to measure the quality of our image and video generative models (text to image, image to video, text to video, etc.) in a variety of quality axis, including visual quality, prompt following, identity preservation, naturalness, and visual text generation and so on. To this end, we require the support of 10 additional data analysts, 5 for image in Menlo Park, (CA) to work closely with the team to prototype and iterate on the measurements before they are scaled up.
Requirements
- Associate's degree or equivalent training required in Computer Science, Electronic Engineering, Physics, Bioinformatics, or other STEM subjects.
- Verbal and written communication skills, problem solving skills, and interpersonal skills.
- Attention to details and an aptitude to experimental investigations
- Basic ability to work independently and manage one’s time.
- Basic knowledge of Python, Unix, and SQL.
- Basic knowledge of computer vision and generative models
- Be onsite, in MPK, CA working with engineers
Nice To Haves
- Prior industrial experience in software development and testing and / or research experience in human computer interaction are preferred.
Responsibilities
- Working with the team to prototype human rating and semi-automatic model evaluation protocols.
- For high stake and / or high complexity protocols, conduct small-to-medium scale in-house human rating on generated content.
- Working with engineers to identify and summarize model gaps based on the above evaluation
- Working with engineers to identify data needed to close those gaps and mine such data to prepare training data for the next iteration of model training
- Working with PDO teams to scale up validated evaluation protocols, including coordination and auditing.
Benefits
- 1099 – No Benefits
- Temp hourly employee benefits, if scheduled to work at least 30 hours per week: medical, dental, vision, 401k.
- Temp salaried employee benefits, if scheduled to work at least 30 hours per week: medical, dental, vision, 401k, holidays.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Entry Level
Education Level
Associate degree
Number of Employees
5,001-10,000 employees
