use ssh compression and a ssh-cipher that enables AES-NI

[glaweh]

If the transfer rate is limited by

• network-bandwidth, enable SSH stream compression
• ssh/sshd cpu load, switch the ssh-cipher to a one that utilizes
  hardware-accelerated crypto (like x86-aes-ni)

[vog]

@glaweh Did you make performance measurements to verify that these SSH options actually do what they are supposed to do? If so, what speedup (roughly) did you measure?

[txt-file]

Out of curiosity I just did a performance test on two of my systems. I used openssl speed command I got via https://calomel.org/aesni_ssl_performance.html. I used the 16384 bytes column.

A rented virtual server with AMD EPYC CPU:

• 281 MB/s for aes-128-gcm for openssl speed -elapsed -evp aes-128-gcm
• 4119 MB/s for aes128-128-gcm OPENSSL_ia32cap="~0x200000200000000" openssl speed -elapsed -evp aes-128-gcm
• 2148 MB/s for ChaCha20-Poly1305 openssl speed -elapsed -evp chaha20-poly1305 (it tested that cipher as openssh did chose that cipher for a ssh connection)

On an arm based system:

• 70 MB/s for openssl speed -elapsed -evp aes-128-gcm
• 57 MB/s for OPENSSL_armcap="~0x200000200000000" openssl speed -elapsed -evp aes-128-gcm
• 300 MB/s for openssl speed -elapsed -evp chacha20-poly1305

So changing the default cipher seems like a bad idea. It depends to much on the environment if it makes de-/encryption faster or slower.

I have not tested if compression has any benefit/harm.

[vog]

@txt-file Thanks for testing! So I assume we still might want to use the -C option, but leave the selection of a cypher to the OS settings and/or user settings, trusting they provide sane defaults for their hardware.

[vog]

@txt-file @glaweh What is your opionion on adding -C?

[txt-file]

As always: It depends / Its complicated.

It depends how compressable the input stream is. These test ignore the decompression. According to https://manpages.debian.org/bookworm/openssh-client/ssh.1.en.html ssh uses gzip for compression.

[nix-shell:~/Downloads]$ time cat zero.img > /dev/null
real	0m1.836s
user	0m0.012s
sys	0m0.713s
[nix-shell:~/Downloads]$ time cat random.img > /dev/null
real	0m1.800s
user	0m0.012s
sys	0m0.711s
[nix-shell:~/Downloads]$ time gzip -k -c zero.img > /dev/null
real	0m5.683s
user	0m5.390s
sys	0m0.253s
[nix-shell:~/Downloads]$ time gzip -k -c random.img > /dev/null
real	0m39.580s
user	0m37.488s
sys	0m1.206s
[nix-shell:~/Downloads]$ […some commands to get files instead of writing to STDOUT…]
[nix-shell:~/Downloads]$ ls -lh random.img* zero.img*
-rw-r--r-- 1 txt staff  1.0G Mar 26 15:30 random.img
-rw-r--r-- 1 txt staff  1.1G Mar 26 15:30 random.img.gz
-rw-r--r-- 1 txt staff  1.0G Mar 26 15:08 zero.img
-rw-r--r-- 1 txt staff 1018K Mar 26 15:08 zero.img.gz
[nix-shell:~/Downloads]$ 

Take these tests with a graintablespoon of salt ;-) In these best case gzip is ~2,5 times slower and saves huge amount of data. In the worst case it is ~20 times slower and transfers the same amount of data.

With current network bandwidth gzip is not such a good idea anymore. lz4 would be better but I do not see an option in OpenSSH to set the compression algorithm.

I am very unsure if -C is benefitial for bscp. The OpenSSH devs set compression as disabled by default and I kind of trust them more than my tests. So I would not enable compression for bscp.

[glaweh]

From today's perspective:

• leave ssh compression off
• still use hw-accelerated cipher
• and put zstd and mbuffer into the IO pipe

zstd can (un-)compress with multiple threads (option -T0 would use as many threads as there are cores)