Fast copy with tar and ncat

September 29, 2024

Hot take alert

You don’t need rsync or scp to copy a large amount of data between two machines…

…provided that:

1. You don’t need encryption.
2. Target machine is directly accessible from source machine.

Do that instead:

1. Install tar and ncat on both machines (if they aren’t installed already).
2. Pick a port number. I’m gonna use 34567 here.
3. Open that port in the firewall on the target machine.

For RedHat-based distributions the command likely is:

sudo firewall-cmd --add-port 34567/tcp

For Debian-based distributions the command likely is

sudo ufw allow 34567/udp

4. cd to the target directory on the target machine.
5. Run the following command on the target machine:

ncat -l 34567 | sudo tar xf -

6. cd to the source directory on the source machine.
7. Run the following command on the source machine:

sudo tar cf - . | ncat TARGET_HOST_OR_IP 34567

You can also:

• Add pv in between tar and ncat on one or both ends to see data transfer rate.
• Read tar manual and add flags to preserve SELinux or other attributes.
• Use that approach with cat, dd or something else instead of tar if you transfer something that is not a directory.

Go back to all notes