Hey all, I found the post about Don Knuth and ChatGPT [1] very interesting so I hacked together this project over the weekend. Optimystic periodically re-runs Knuth's questions against the latest GPT model (additionally I've asked GPT to score the updated answers as PASS/FAIL).
Initially I thought it would just be a fun thing, but I've realized there could be some value to the larger LLM community. Also I think it would be interesting to apply this format to experts in other fields.
Comments
Hey all, I found the post about Don Knuth and ChatGPT [1] very interesting so I hacked together this project over the weekend. Optimystic periodically re-runs Knuth's questions against the latest GPT model (additionally I've asked GPT to score the updated answers as PASS/FAIL).
Initially I thought it would just be a fun thing, but I've realized there could be some value to the larger LLM community. Also I think it would be interesting to apply this format to experts in other fields.
Appreciate any feedback.
[1] https://news.ycombinator.com/item?id=36012360