Comment on ArchiveTeam needs OPMLs and feed URLs to grab cached data from Google ReaderparentComments−tmzt13yAny way I could patch the program to stop it from deleting the data after it is uploaded?Among other things I would like to set up an ElasticSearch cluster for my own feeds.Is the WARC format defined somewhere? I haven't looked at any other ArchiveTeam projects so I'm not informed if this format is used elsewhere.−ivankOP13yYeah, you could patch seesaw-kit to not delete local data. Note that greader-grab just gets a random work item from the tracker.There's an ISO spec for WARC and tools linked at http://www.archiveteam.org/index.php?title=The_WARC_Ecosyste...−subsystem13yI haven't tried it, but the --keep-data option might work?https://github.com/ArchiveTeam/seesaw-kit/blob/master/run-pi...−ivankOP13yYeah, that looks like the right thing.
Comments
Any way I could patch the program to stop it from deleting the data after it is uploaded?
Among other things I would like to set up an ElasticSearch cluster for my own feeds.
Is the WARC format defined somewhere? I haven't looked at any other ArchiveTeam projects so I'm not informed if this format is used elsewhere.
Yeah, you could patch seesaw-kit to not delete local data. Note that greader-grab just gets a random work item from the tracker.
There's an ISO spec for WARC and tools linked at http://www.archiveteam.org/index.php?title=The_WARC_Ecosyste...
I haven't tried it, but the --keep-data option might work?
https://github.com/ArchiveTeam/seesaw-kit/blob/master/run-pi...
Yeah, that looks like the right thing.