To be fair, here is a tool that has features no other tool I know: https://github.com/rlmcpherson/s3gof3r
It can saturate your bandwidth and you can pipe to/from it.
It's great to deal with hundreds GB's archive on the fly.
"Examples:
$ tar -cf - /foo_dir/ | gof3r put -b my_s3_bucket -k bar_dir/s3_object
$ gof3r get -b my_s3_bucket -k bar_dir/s3_object | tar -x
Comments
How does its performance compare to aws s3 sync? Note that you can adjust max_concurrent_requests which defaults to 10.
I'm always surprised by the number of "download from s3" tools there are. AWS-CLI does a really quite nice job of it.
To be fair, here is a tool that has features no other tool I know: https://github.com/rlmcpherson/s3gof3r It can saturate your bandwidth and you can pipe to/from it. It's great to deal with hundreds GB's archive on the fly. "Examples: $ tar -cf - /foo_dir/ | gof3r put -b my_s3_bucket -k bar_dir/s3_object $ gof3r get -b my_s3_bucket -k bar_dir/s3_object | tar -x
It's written in golang. So it's a simple binary.