Comment on Hacker News SearchparentComments−revorad15ySo is HN data now being stored in ThriftDB? Or is it being replicated on your server? If so, what's the lag?−andres15yWe wrote a crawler to download data from HN, parse it into JSON, and upload it to ThriftDB. The webapp at hnsearch.com sends requests directly to ThriftDB. There's a ~15min lag between HN and HNSearch.
Comments
So is HN data now being stored in ThriftDB? Or is it being replicated on your server? If so, what's the lag?
We wrote a crawler to download data from HN, parse it into JSON, and upload it to ThriftDB. The webapp at hnsearch.com sends requests directly to ThriftDB. There's a ~15min lag between HN and HNSearch.