They had access only to just one tiny bit of the entire written words of the worlds. Not all books are available in electronic formats. You can see from the visualization in the article that we don’t even have the titles for a lot of published books with ISBN. And even so, books with ISBN comprise only a fraction of the entire ever written books. Not too mention books in various (minor) languages of the worlds.
Comments
Since most foundational models have been trained on illegally acquired books, this info should be already baked in.
They had access only to just one tiny bit of the entire written words of the worlds. Not all books are available in electronic formats. You can see from the visualization in the article that we don’t even have the titles for a lot of published books with ISBN. And even so, books with ISBN comprise only a fraction of the entire ever written books. Not too mention books in various (minor) languages of the worlds.