Haha yeah, the site automatically assigns the term to any benchmark that fewer than 25% of the tested models are able to pass.
What’s more surprising to me is that, unlike the “pelican riding a bicycle” whose objectivity has been slightly compromised as newer models have incorporated it into their training data, the arbitrary-point star has been wiping models out ever since the early days of Flux back in 2024.
I personally love the test because it's something that even an elementary school child with no artistic experience at all can do, but state of the art models struggle heavily.
Comments
Haha yeah, the site automatically assigns the term to any benchmark that fewer than 25% of the tested models are able to pass.
What’s more surprising to me is that, unlike the “pelican riding a bicycle” whose objectivity has been slightly compromised as newer models have incorporated it into their training data, the arbitrary-point star has been wiping models out ever since the early days of Flux back in 2024.
I personally love the test because it's something that even an elementary school child with no artistic experience at all can do, but state of the art models struggle heavily.