Maybe you should take another look at pricing? DynamoDB is like $0.002/GB/month more expensive than S3 for storage. Requests are more expensive, but you're also not reinventing every wheel and if they're as infrequent a set of requests as this seems designed for it's still only a few bucks a month.
Comments
How is this different than AWS Athena ? - https://aws.amazon.com/blogs/aws/amazon-athena-interactive-s...
Also why don't you dump the log data into a NoSQL like dynamoDB instead of S3 ?
Athena looks cool. Didn't know about it. It probably describes what I'm trying to do.
Price.
Maybe you should take another look at pricing? DynamoDB is like $0.002/GB/month more expensive than S3 for storage. Requests are more expensive, but you're also not reinventing every wheel and if they're as infrequent a set of requests as this seems designed for it's still only a few bucks a month.
My infrequent reads and writes are huge, and possibly spikey.
As I understand it, for $4.68/month I can only add 36 MB/hour, and thats assuming my objects are in exact multiple of 1KB.
Ahh, gotcha. That makes sense, this setup makes a lot more sense if you're dealing with very spikey reads.
If your data storage requirement is semi-constant at 1.5TB and is ephemeral, have you looked at just using your own replicated servers ?
That would probably turn out to be cheaper than using S3 - https://github.com/minio/minio
The data is not ephemeral, only the cache is.