We did NBD in preference to everything else because NBD is so simple, and is the kernel's de facto answer to "abstract a block device away from the hardware". Shaun ran into reliability issues with it (which, for all I know, were just bugs fixed in more recent kernels) and iSCSI was within arm's reach, so it wound up being iSCSI.
We could do a bunch of testing and find the performance gains of NVMe-TCP worth the switch-out, and that project would be tenable (though: big enough we wouldn't do it unless the win was really big).
A serious NVMe-OF deployment would be tantamount to us building an EBS-style SAN scheme, which almost certainly would require us to single out specific regions to get the "better"/"newer" disk storage in, which is something we haven't had to do yet. I think in the next year or two we're more interested in seeing how LSVD, and especially Tigris LSVD, plays out.
Comments
We did NBD in preference to everything else because NBD is so simple, and is the kernel's de facto answer to "abstract a block device away from the hardware". Shaun ran into reliability issues with it (which, for all I know, were just bugs fixed in more recent kernels) and iSCSI was within arm's reach, so it wound up being iSCSI.
We could do a bunch of testing and find the performance gains of NVMe-TCP worth the switch-out, and that project would be tenable (though: big enough we wouldn't do it unless the win was really big).
A serious NVMe-OF deployment would be tantamount to us building an EBS-style SAN scheme, which almost certainly would require us to single out specific regions to get the "better"/"newer" disk storage in, which is something we haven't had to do yet. I think in the next year or two we're more interested in seeing how LSVD, and especially Tigris LSVD, plays out.