I'm aware of the comments saying rsync or rclone, and I absolutely also use those, but for initial migrations I often find myself simply using tar instead.
Something like :
(cd /src/dir && tar cf -) | ssh user@host "(cd /dst/dir && tar xvf -)"
will do nicely.
It won't saturate 4 multichannel links, you'll most likely need rclone or rsync for that (or robocopy it appears), but it's much faster than rsync on initial copies.
Then for good measure, once the tar copy finishes, I run a single rsync verification between the two targets.
That being said, my latest storage migration was performed using either ChronoSync or Carbon Copy Cloner from my Mac mini server. They won't do anything rsync cannot already do, except present data and copy history in a graphical environment.
For large transfers on a network I control I would use bare netcat, much faster than going over ssh and without having to figure out rsync server (probably easy, but I don't need another distraction.
nc -l 8888 | tar -C /dir -xzf -
tar -C /dir -czf - | nc server 8888
rsync however, is totally worth it if there is any chance the transfer gets interrupted.
Be aware that tar doesn't preserve creation/birth times and keeping that original metadata is important to some people if the target file system can record it. Robocopy preserves/transfers creation times by default and recent rsync versions (3.2.3 2020-08-06) can do it with the extra option --crtimes.
Comments
I'm aware of the comments saying rsync or rclone, and I absolutely also use those, but for initial migrations I often find myself simply using tar instead.
Something like :
will do nicely.It won't saturate 4 multichannel links, you'll most likely need rclone or rsync for that (or robocopy it appears), but it's much faster than rsync on initial copies.
Then for good measure, once the tar copy finishes, I run a single rsync verification between the two targets.
That being said, my latest storage migration was performed using either ChronoSync or Carbon Copy Cloner from my Mac mini server. They won't do anything rsync cannot already do, except present data and copy history in a graphical environment.
For large transfers on a network I control I would use bare netcat, much faster than going over ssh and without having to figure out rsync server (probably easy, but I don't need another distraction.
rsync however, is totally worth it if there is any chance the transfer gets interrupted.>simply using tar instead
Be aware that tar doesn't preserve creation/birth times and keeping that original metadata is important to some people if the target file system can record it. Robocopy preserves/transfers creation times by default and recent rsync versions (3.2.3 2020-08-06) can do it with the extra option --crtimes.