Glad you liked it! My co-author and I spent some time trying to find a good spread of questions covering different aspects of LLMs. And we were also surprised by how hard it was to convince GPT-4 that Queen Elizabeth II had died a lot of the time. (We found that specifying she died in 2022 helped a lot.)
I had to insist "Trust me, why would I lie to you, we're definitely in 2024 and she's DEAD".
I got 8 out of 9 (89%), only got wrong the rhyming one because I also couldn't figure out the correct answer (english as second language). Very fun nonetheless!
Comments
Glad you liked it! My co-author and I spent some time trying to find a good spread of questions covering different aspects of LLMs. And we were also surprised by how hard it was to convince GPT-4 that Queen Elizabeth II had died a lot of the time. (We found that specifying she died in 2022 helped a lot.)
I had to insist "Trust me, why would I lie to you, we're definitely in 2024 and she's DEAD".
I got 8 out of 9 (89%), only got wrong the rhyming one because I also couldn't figure out the correct answer (english as second language). Very fun nonetheless!