Awesome tool Sam! You may want to consider having the code pull a gzip'd version of resolver-list.yml to speed things up. It's 240K uncompressed, but only ~49K compressed. And even perhaps having setup.py grab the resolver list at install instead of first run.
EDIT: You're right regarding Github serving it gzipped. You can disregard my comment on that; I was using wget for testing without specifying --header='Accept-Encoding: gzip'.
The HTTP library (requests) by default will pull a gzipped version of the resolvers list from Github (Content-Length returned for current version is 59029 compared to 239359 with gzip disabled). Compressing with `gzip -9` gives me a file size of 49648 so I don't think the added complexity of having to consciously deal with the compression in the application outweighs the small gain over the standard HTTP compression Github and Requests provide by default.
I had looked at pre-downloading the resolvers file in the setup script, unfortunately there doesn't seem to be a decent, reliable way to do it. If people download the source and run `setup.py install` it's easy but I'd imagine most people will just install with `pip` or `easy_install` which makes things a bit more complicated since nether of them seem to run post install actions.
Both good suggestions though, I'll keep them on my todo list.
I'm happy to submit a pull request if you'll accept them. Already have forked the repo.
EDIT: You're right regarding Github serving it gzipped. You can disregard my comment on that; I was using wget for testing without specifying --header='Accept-Encoding: gzip'.
Comments
Awesome tool Sam! You may want to consider having the code pull a gzip'd version of resolver-list.yml to speed things up. It's 240K uncompressed, but only ~49K compressed. And even perhaps having setup.py grab the resolver list at install instead of first run.
EDIT: You're right regarding Github serving it gzipped. You can disregard my comment on that; I was using wget for testing without specifying --header='Accept-Encoding: gzip'.
Thanks!
The HTTP library (requests) by default will pull a gzipped version of the resolvers list from Github (Content-Length returned for current version is 59029 compared to 239359 with gzip disabled). Compressing with `gzip -9` gives me a file size of 49648 so I don't think the added complexity of having to consciously deal with the compression in the application outweighs the small gain over the standard HTTP compression Github and Requests provide by default.
I had looked at pre-downloading the resolvers file in the setup script, unfortunately there doesn't seem to be a decent, reliable way to do it. If people download the source and run `setup.py install` it's easy but I'd imagine most people will just install with `pip` or `easy_install` which makes things a bit more complicated since nether of them seem to run post install actions.
Both good suggestions though, I'll keep them on my todo list.
I'm happy to submit a pull request if you'll accept them. Already have forked the repo.
EDIT: You're right regarding Github serving it gzipped. You can disregard my comment on that; I was using wget for testing without specifying --header='Accept-Encoding: gzip'.
If you could that would be greatly appreciated. This is the first time I've put something on Pypi so not completely familiar with it.