Quality measurement remains challenging for generative AI due to the lack of reliable and robust technical metrics, especially when minor differences in generated details significantly impact user experience. iSoftStone, Inc. is seeking Data Analysts to prototype and iterate on measurements for image and video generative models.
Requirements
- Basic knowledge of Python, Unix, and SQL.
- Basic knowledge of computer vision and generative models
Responsibilities
- Working with the team to prototype human rating and semi-automatic model evaluation protocols.
- For high stake and / or high complexity protocols, conduct small-to-medium scale in-house human rating on generated content.
- Working with engineers to identify and summarize model gaps based on the above evaluation
- Working with engineers to identify data needed to close those gaps and mine such data to prepare training data for the next iteration of model training
- Working with PDO teams to scale up validated evaluation protocols, including coordination and auditing.
Other
- Associate's degree or equivalent training required in Computer Science, Electronic Engineering, Physics, Bioinformatics, or other STEM subjects.
- Prior industrial experience in software development and testing and / or research experience in human computer interaction are preferred.
- Verbal and written communication skills, problem solving skills, and interpersonal skills.
- Attention to details and an aptitude to experimental investigations
- Basic ability to work independently and manage one’s time.
- Be onsite, in MPK, CA working with engineers
