I didn't include it in my summary (it took me an hour to read the whole thing, obviously a lot had to be cut) but the article does actually address the "high resolution" argument in a three-paragraph bullet point under the "Sample Inefficiency" subheading: https://gwern.net/llm-catapult#sample-inefficiency If you read it on a 4K screen at 120 FPS, you should be able to take in its information content in less than a microsecond.
They "address" it by making false statement that the video stream is highly predictable. Sure, you might be able to predict 99% of video stream (for which you'd need to have a physics model, negating the whole point of baby fast learning), but the remaining 1% is still in terabytes if not petabytes per year.
Comments
I didn't include it in my summary (it took me an hour to read the whole thing, obviously a lot had to be cut) but the article does actually address the "high resolution" argument in a three-paragraph bullet point under the "Sample Inefficiency" subheading: https://gwern.net/llm-catapult#sample-inefficiency If you read it on a 4K screen at 120 FPS, you should be able to take in its information content in less than a microsecond.
They "address" it by making false statement that the video stream is highly predictable. Sure, you might be able to predict 99% of video stream (for which you'd need to have a physics model, negating the whole point of baby fast learning), but the remaining 1% is still in terabytes if not petabytes per year.