Skip to content

Comment on The Pile: An 800GB dataset of diverse text for language modeling (2020)parent

Comments

It might seem like I’m entrenched in my position, but it’s quite the opposite — the only reason I’m doing this is because I really believe it’s the best outcome for devs in the long run. I’m open to changing my mind and pulling the plug in everything.

I’ll read over your essay and give it some thought. There are a bunch of subtle aspects to consider; I’ve been thinking it over for about four months now and still haven’t covered all the territory yet.

It feels like this may be one of the most important decisions going forward — both from an intellectual property point of view, and an individual rights perspective. E.g. you mention that it’s civil disobedience to share the weights, but it feels like if someone is claiming to do open science (LLaMA), sharing the research materials is the minimum requirement. Plus look how it’s benefited them; they’ve captured most of the open source LLM mindshare. So it seems likely that this will lead to more open source work in the long run, not less.

Feel free to chat! You can DM me on Twitter or email me. I’ve been in the hospital with my wife for 7 weeks, with two to go, so I’ve been a bit less responsive than I usually am.

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.