This whole article is a long-winded way of saying, "Hey, nerds! The gravy train's over! GPUs aren't free and Bill Gates was right! You can't eat freedom[0]! Either you point a gun at your users' temples or you starve to death!"
In some sense this is right, at least as long as all the research we're replicating boils down to "hey if you double the GPUs and data you get double the model quality". But whenever you make the "shit costs money" argument, you also entertain a moralistic co-argument: "I am entitled to supracompetitive profit". And this guy is definitely on the side of "pay up, capitalism works, you pinko commies".
The problem with this argument is that the datasets we currently train on are already stolen data. With the exception of extant FOSS licensing, which probably allow training[1], of course. The argument for closed models boils down to "we are entitled to be paid for our model but the artists and writers we stole[2] from are not".
[0] The clarion call of any savvy fascist
[1] Good luck figuring out how to comply with CC-BY-SA though
[2] In the Valentian sense where recording a movie onto a VHS tape makes you a home-invading rapist.
Comments
This whole article is a long-winded way of saying, "Hey, nerds! The gravy train's over! GPUs aren't free and Bill Gates was right! You can't eat freedom[0]! Either you point a gun at your users' temples or you starve to death!"
In some sense this is right, at least as long as all the research we're replicating boils down to "hey if you double the GPUs and data you get double the model quality". But whenever you make the "shit costs money" argument, you also entertain a moralistic co-argument: "I am entitled to supracompetitive profit". And this guy is definitely on the side of "pay up, capitalism works, you pinko commies".
The problem with this argument is that the datasets we currently train on are already stolen data. With the exception of extant FOSS licensing, which probably allow training[1], of course. The argument for closed models boils down to "we are entitled to be paid for our model but the artists and writers we stole[2] from are not".
[0] The clarion call of any savvy fascist
[1] Good luck figuring out how to comply with CC-BY-SA though
[2] In the Valentian sense where recording a movie onto a VHS tape makes you a home-invading rapist.