It depends. Taxpayer dollars may only barely cover the cost of getting the data together, but distributing it could be fairly significant as a budgetary item. To cover that cost they may just charge some fee that has several uses:
- Keep taxes down and only support people who really have a serious need for it
- Cover the cost of packaging and distribution (and other associated costs)
I don't necessarily agree with this, but these are the kinds of justifications that are generally tossed around inside the government about these kinds of things.
A better question for the people that live there is 'are the cost of distributing the data artificially inflated?'.
In my own county the costs of document scanning and record storage were in the 1 to 2 million dollar range for years. Oddly this coincided with the county commissioner having a husband that work for Xerox at the time, the Xerox office also handled the counties scanning and document storage. Amazingly after the time that commissioner was no longer in the office the contracts were renegotiated and the price dropped over 25%. The biggest issue I have seen working at the local government level for over a decade now is very few people without a stake in the game ever review the use of technology by the counties. Politics has far more sway than best practices.
I'm assuming you meant the prices dropped to 1/4 of their previous level. Even then, pretty much any civic minded organization or "citizen scientist" is priced out of the market. In fact, the only people who can access it, in that range, are people who can turn a profit on it, i.e., likely not looking out for the public interest as a primary concern.
Prices didn't drop, but oddly enough they were not excessively priced in the first place, as compared to many other counties. The operations of the records department are funded from the general fund and not self supporting. Still if you were purchasing many thousands of pages like someone in the data sciences would, the cost would be excessive.
In retrospect probably one of the dumber things I've ever done is not keep 2TB of county records from 1850 to 2008 after a company closed down operations. There wasn't any particular reason I couldn't keep them, they were paid for and the company didn't sell them off after closing. The server was likely neglected for years after that and eventually formatted.
In a roundabout way, that answers the original question, doesn't it? This sort of data is so seldom used, that efforts at 'standardizing' and 'broad distribution' don't scale. You didn't see any value or use in the data for years, only in hindsight, maybe, it would have been fun to look at. Because that's what it is - it's fun (for those from outside the field) to read posts like the OP every once in a while, but there's very little actual value in it, and it's too much work for the vast majority of people to do themselves.
I work in geospatial modelling and I use data like this every day; have been for 15 years. So I know a thing or two about availability of this sort of data, how hard it is to assemble and maintain, and how many people actually do something with it. In the EU, there are directives that create obligations for governments to offer all sorts of data freely to everybody. The amount of money this costs is staggering, and although I'm very happy with it (because it has made my life 100 times easier in the data respect), the useful stuff that is done with it by people who didn't have access to it before is minimal; and certainly disproportional to the amount of money that goes into it. The consultants are laughing all the way to the bank, of course (and hey, I get paid indirectly from it as well, so it's not like I'm complaining).
So that makes the 'data should be open' mantra an entirely ideological disposition. I used to believe it as well, but then why shouldn't we also say 'governments can only spend money on vegan stuff because some people believe in 'animals rights'?' I no longer see pet peeves like 'free data' as black and white as I used to. Bureaucracies are expensive, partly because their goal is not efficiency (which is something that is usually overlooked by the naive) and partly because of other effects inherent to them. We need to restrain ourselves in what and how much we demand from them.
Comments
It depends. Taxpayer dollars may only barely cover the cost of getting the data together, but distributing it could be fairly significant as a budgetary item. To cover that cost they may just charge some fee that has several uses:
- Keep taxes down and only support people who really have a serious need for it
- Cover the cost of packaging and distribution (and other associated costs)
I don't necessarily agree with this, but these are the kinds of justifications that are generally tossed around inside the government about these kinds of things.
A better question for the people that live there is 'are the cost of distributing the data artificially inflated?'.
In my own county the costs of document scanning and record storage were in the 1 to 2 million dollar range for years. Oddly this coincided with the county commissioner having a husband that work for Xerox at the time, the Xerox office also handled the counties scanning and document storage. Amazingly after the time that commissioner was no longer in the office the contracts were renegotiated and the price dropped over 25%. The biggest issue I have seen working at the local government level for over a decade now is very few people without a stake in the game ever review the use of technology by the counties. Politics has far more sway than best practices.
I'm assuming you meant the prices dropped to 1/4 of their previous level. Even then, pretty much any civic minded organization or "citizen scientist" is priced out of the market. In fact, the only people who can access it, in that range, are people who can turn a profit on it, i.e., likely not looking out for the public interest as a primary concern.
Prices didn't drop, but oddly enough they were not excessively priced in the first place, as compared to many other counties. The operations of the records department are funded from the general fund and not self supporting. Still if you were purchasing many thousands of pages like someone in the data sciences would, the cost would be excessive.
In retrospect probably one of the dumber things I've ever done is not keep 2TB of county records from 1850 to 2008 after a company closed down operations. There wasn't any particular reason I couldn't keep them, they were paid for and the company didn't sell them off after closing. The server was likely neglected for years after that and eventually formatted.
In a roundabout way, that answers the original question, doesn't it? This sort of data is so seldom used, that efforts at 'standardizing' and 'broad distribution' don't scale. You didn't see any value or use in the data for years, only in hindsight, maybe, it would have been fun to look at. Because that's what it is - it's fun (for those from outside the field) to read posts like the OP every once in a while, but there's very little actual value in it, and it's too much work for the vast majority of people to do themselves.
I work in geospatial modelling and I use data like this every day; have been for 15 years. So I know a thing or two about availability of this sort of data, how hard it is to assemble and maintain, and how many people actually do something with it. In the EU, there are directives that create obligations for governments to offer all sorts of data freely to everybody. The amount of money this costs is staggering, and although I'm very happy with it (because it has made my life 100 times easier in the data respect), the useful stuff that is done with it by people who didn't have access to it before is minimal; and certainly disproportional to the amount of money that goes into it. The consultants are laughing all the way to the bank, of course (and hey, I get paid indirectly from it as well, so it's not like I'm complaining).
So that makes the 'data should be open' mantra an entirely ideological disposition. I used to believe it as well, but then why shouldn't we also say 'governments can only spend money on vegan stuff because some people believe in 'animals rights'?' I no longer see pet peeves like 'free data' as black and white as I used to. Bureaucracies are expensive, partly because their goal is not efficiency (which is something that is usually overlooked by the naive) and partly because of other effects inherent to them. We need to restrain ourselves in what and how much we demand from them.