It seems like there needs to be a real break between the collectors of data and the analyzers. At the moment heaps of data is collected by the same teams that analyze it, but just as an API opens up a codebase to new ideas, so would a standard practice of placing the data into a library before starting analysis.
Fortunately there's already a growing number of places (typically government in my experience) making an effort just to keep data libraries up and open to further research.
I agree that this is a problem, but I fear it's not as easy to solve as with traditional libraries that store books. For data to be useful you need a lot of very formal high quality metadata, and that metadata has to be collected by the original researchers as part of their process. So I think there has to be an incentive or mandate that makes researchers collect and document data in a more systematic way. Librarians cannot do it for them.
Comments
It seems like there needs to be a real break between the collectors of data and the analyzers. At the moment heaps of data is collected by the same teams that analyze it, but just as an API opens up a codebase to new ideas, so would a standard practice of placing the data into a library before starting analysis.
Fortunately there's already a growing number of places (typically government in my experience) making an effort just to keep data libraries up and open to further research.
I agree that this is a problem, but I fear it's not as easy to solve as with traditional libraries that store books. For data to be useful you need a lot of very formal high quality metadata, and that metadata has to be collected by the original researchers as part of their process. So I think there has to be an incentive or mandate that makes researchers collect and document data in a more systematic way. Librarians cannot do it for them.