All of the above would work, but feels less than ideal. I'm pretty comfortable using a well-schema'd relational database to manage data, but I don't think it fits something I'm working on atm.
I'm collecting and parsing data from a few different types of sources (think: some web page scrapers, Twitter, RSS feeds) for later analysis. I want an intermediate data store where I can throw all of the data together for querying in the short term (within days).
Some of the features I extract from it will probably be stored for longer-term use in a regular database. The JSON itself I expect won't ever be referred to in the long term.
If I can think of a new piece of data I might want to look at, it's very appealing to be able to just print it out in one of the data-gathering programs, without having to touch the entire stack top-to-bottom, deploy a new schema, etc.
Comments
All of the above would work, but feels less than ideal. I'm pretty comfortable using a well-schema'd relational database to manage data, but I don't think it fits something I'm working on atm.
I'm collecting and parsing data from a few different types of sources (think: some web page scrapers, Twitter, RSS feeds) for later analysis. I want an intermediate data store where I can throw all of the data together for querying in the short term (within days).
Some of the features I extract from it will probably be stored for longer-term use in a regular database. The JSON itself I expect won't ever be referred to in the long term.
If I can think of a new piece of data I might want to look at, it's very appealing to be able to just print it out in one of the data-gathering programs, without having to touch the entire stack top-to-bottom, deploy a new schema, etc.