The application automatically opens in a new Chrome window.
From here, I grabbed a data dump from one of our external providers.
We work with a lot of providers who are really technologically challenged. I'd love to be able to say, here you are.. here is our API, start pushing your content to us. But in practice they don't even know what their XML feeds do. We need their data, but getting a consistent dataset from them when they seem to change their format regularly is a pain! And when importing only 10 or so items at a time it's excruciatingly painful.
Today I learnt how easy that can be with Google Refine!
Comments
I found this really helpful. Not so much to do with the actual article, but to do with actual making me aware of Google Refine.
Installation was a breeze. I couldn't find any instructions, but it was as simple as downloading for Linux, extracting, the running the shell script.
http://code.google.com/p/google-refine/downloads/detail?name...
The application automatically opens in a new Chrome window.
From here, I grabbed a data dump from one of our external providers.
We work with a lot of providers who are really technologically challenged. I'd love to be able to say, here you are.. here is our API, start pushing your content to us. But in practice they don't even know what their XML feeds do. We need their data, but getting a consistent dataset from them when they seem to change their format regularly is a pain! And when importing only 10 or so items at a time it's excruciatingly painful.
Today I learnt how easy that can be with Google Refine!
Also check out Data Wrangler. http://vis.stanford.edu/wrangler/
It focuses on more mechanical transformations but has the ability to save the steps to a program which you can then use in a process pipeline.
(disclaimer, I haven't played with it in a few months so this is from memory)
GR also allows you to export/import steps to reuse. Though I think DataWrangler is easier to integrate in to an automated pipeline.
Yeah, Wrangler will turn your "manipulation" steps into scripts in Python/MapReduce/Javascript.
I'm glad you found it useful Richard.
Tip: GR can have a bit of a wobble from time-to-time, usually restarting the process will sort things out.