EFS will handle 1000 readers/writers. We tested it as a data exchange medium for computational tasks. The meta-information APIs in EFS in my experience are faster than S3's (LIST in S3 is notorious). The overall amount of data we stored in EFS was pretty limited (single-digit terabytes), though.
I wouldn't use EFS to store petabytes of data, but if you need a resilient and scalable storage that you can easily integrate into your application, then EFS is great.
One thing that I loved, is the ease of use in local development. With EFS you can simply mount the shared volume into your Docker/K8s container in production, and a local directory when you're developing tasks locally on your laptop. You can even run tasks without a container and monitor their output by looking at the exchange directory. There are AWS API emulators (e.g. Localstack) but they are not as convenient.
Comments
EFS will handle 1000 readers/writers. We tested it as a data exchange medium for computational tasks. The meta-information APIs in EFS in my experience are faster than S3's (LIST in S3 is notorious). The overall amount of data we stored in EFS was pretty limited (single-digit terabytes), though.
I wouldn't use EFS to store petabytes of data, but if you need a resilient and scalable storage that you can easily integrate into your application, then EFS is great.
One thing that I loved, is the ease of use in local development. With EFS you can simply mount the shared volume into your Docker/K8s container in production, and a local directory when you're developing tasks locally on your laptop. You can even run tasks without a container and monitor their output by looking at the exchange directory. There are AWS API emulators (e.g. Localstack) but they are not as convenient.