For every participant we maintain the following in the leaderboard:
Bonus Quota: We multiply the number of Adverse Examples a participant has discovered by 5 in order to calculate the bonus quota per participant. In other words, for every example that is scored as adverse the participant will be allowed to submit 5 more image-label pairs.
Human verification: Human raters will be rating all the image-label pairs in the submission Queue by the participant continuously throughout the challenge.
Awarding points: If multiple participants submit the same image-label pair, a point is awarded to the first participant who submits it (based on the timestamp in the submitted images queue)
Adverse examples: Image-label pairs for which the human verification is in disagreement with a machine prediction, e.g.
Winner: The winner is the participant with the highest number of Adverse Examples when the competition closes.
Check Rules to learn the challenge conditions.
Check Participate to learn how to start contributing.