This project immediately won my heart by starting off with regular expressions. It amazes me how many coders flip out at the suggestion of teaching/advocating regexes to novices, as if the first thing a novice will attempt to do is parse HTML with regex or build a credit card validation application.
I've spent most of my life in the journalism world, and I imagine parts of it, especially the investigative projects time, share some characteristics with the law world...in that you're dealing with messy documents and data. You would not f--king believe how much time is wasted on manually reading lists of names/addresses, because the person has no idea how to split "Smith, Bob -- 1020 Broadway, NY" into a spreadsheet. Nevermind the countless hours spent hand fixing whitespace or other formatting errors, things that could take seconds with even just Sublime Text.
Regexes are amazing because they teach people how to think in patterns, and that manually defining the pattern for your specific domain can be a very powerful thing. I've greatly increased the speed that I can research things across large document sets...simply by grepping documents for patterns that fit proper nouns, or even large money values (for example, /(?:\d{3},){2,}/)...when dealing with messily OCR'ed documents, in which phrases/words will have common mistranslations, being able to grep in order to find a key name/number/noun is incredibly powerful.
IMO, regex is exactly the first thing you should be teaching people like lawyers and researchers. It's a very powerful programming concept, but besides that, it is immediately useful to anyone who can use a text editor. And it opens the door to programming in general (because find-and-replace with a text editor gets old, you'll soon find the need to run grep from the CLI)
Comments
This project immediately won my heart by starting off with regular expressions. It amazes me how many coders flip out at the suggestion of teaching/advocating regexes to novices, as if the first thing a novice will attempt to do is parse HTML with regex or build a credit card validation application.
I've spent most of my life in the journalism world, and I imagine parts of it, especially the investigative projects time, share some characteristics with the law world...in that you're dealing with messy documents and data. You would not f--king believe how much time is wasted on manually reading lists of names/addresses, because the person has no idea how to split "Smith, Bob -- 1020 Broadway, NY" into a spreadsheet. Nevermind the countless hours spent hand fixing whitespace or other formatting errors, things that could take seconds with even just Sublime Text.
Regexes are amazing because they teach people how to think in patterns, and that manually defining the pattern for your specific domain can be a very powerful thing. I've greatly increased the speed that I can research things across large document sets...simply by grepping documents for patterns that fit proper nouns, or even large money values (for example, /(?:\d{3},){2,}/)...when dealing with messily OCR'ed documents, in which phrases/words will have common mistranslations, being able to grep in order to find a key name/number/noun is incredibly powerful.
IMO, regex is exactly the first thing you should be teaching people like lawyers and researchers. It's a very powerful programming concept, but besides that, it is immediately useful to anyone who can use a text editor. And it opens the door to programming in general (because find-and-replace with a text editor gets old, you'll soon find the need to run grep from the CLI)