I'm not sure it would interact nicely with the request authentication system that S3 uses.
I think a generic cache layer would be a better solution. A bit of googling turns up fuse-cache, which sounds like about the right thing (although I haven't actually examined it in detail), or fs-cache, which also sounds like a discrete cache layer to be added to any file system. Basically, you mount it, and it passes requests through to any other mount (like s3fslite) while adding an on-disk cache layer.
I haven't tested any of these, but that seems like an approach worth pursuing.
Comments
Would it be possible to reuse an existing HTTP caching solution like squid or nginx for the caching since s3 exposes a REST api?
I'm not sure it would interact nicely with the request authentication system that S3 uses.
I think a generic cache layer would be a better solution. A bit of googling turns up fuse-cache, which sounds like about the right thing (although I haven't actually examined it in detail), or fs-cache, which also sounds like a discrete cache layer to be added to any file system. Basically, you mount it, and it passes requests through to any other mount (like s3fslite) while adding an on-disk cache layer.
I haven't tested any of these, but that seems like an approach worth pursuing.
- Russ