Hello, this might be related to https://fedoraproject.org/wiki/Changes/OpenSSH_Server_Crypto_Policy Can you check what ciphers, macs were used in your Fedora 26 and what is used now? We can probably blame the change of the default cipher, from ChaCha20 to AES256, but I don't have the exact numbers. Can you try to uncomment last line of /etc/sysconfig/sshd to overwrite system-wide policy: CRYPTO_POLICY= If it has so significant performance impact, we should adjust the policy to prefer ChaCha20. Regards, Jakub Jelen On Thu, 2017-11-16 at 19:44 +0100, Kasper Pedersen wrote: > I have been poking at this for some hours, and by now I could use > a second opinion. Or clue. Thirteen black candles? > > > Somehow, when I sit on a F27 machine and pull from a F26 machine > using > rsync, something extremely inefficient happens in the F26 machine. > > If I sit on a F26 machine and pull from the F26 machine, it is fast. > > > The good state: > --------------- > I had 3 reasonable machines, all on F26: > > Server - 1.8Ghz Haswell-celeron dualcore > Desktop - 3.2GHz Haswell > Backup - 2.4GHz Core2Q > > Backup runs rsync, and pulls from Server and Desktop using ssh. > Nothing fancy, it copies and shuts itself down. > When Backup was reading from Server, Server would have about > > - 30% cpu to rsync > - 60% cpu to ssh > > and deliver 100MB/s reliably. So does Desktop. > > > The bad state: > -------------- > After one such rsync run, I upgraded Backup to F27. > When Backup pulls from Server, Server shows: > > - 99% cpu to rsync > - 6% cpu to ssh > > and it delivers **11MB/s**, one tenth of what it did before. > So, 30x higher cpu load per byte delivered. > > So, somehow, the rsync running on F27, which by the way reports > the same version and capabilities as the one on F26, triggers > something in the spawned process that behaves differently. > > > > On the server, the ssh-spawned rsync process looks the same whether > it is Backup or Desktop pulling from it: > > On Server, with Backup as client, 11MB/s: > root 25042 34.4 0.0 124996 5620 ? Rs 18:54 0:40 > rsync > --server --sender -vlogDtpre.iLsfxC . > /bulk/mux/media/xbn-2017_10_29_010103.m.ts > /bulk/mux/media/xbn-2017_11_04_235600.m.ts > /bulk/mux/media/xbn-2017_11_12_005938.m.ts > /bulk/mux/media/xbn-2017_11_12_011442.m.ts > /bulk/mux/media/xbn-2017_11_12_015031.m.ts > > > On server, with Desktop as client, 100MB/s: > root 25152 31.9 0.0 122344 3000 ? Rs 18:56 0:05 > rsync > --server --sender -vlogDtpre.iLsfxC . > /bulk/mux/media/xbn-2017_10_29_010103.m.ts > /bulk/mux/media/xbn-2017_11_04_235600.m.ts > /bulk/mux/media/xbn-2017_11_12_005938.m.ts > /bulk/mux/media/xbn-2017_11_12_011442.m.ts > /bulk/mux/media/xbn-2017_11_12_015031.m.ts > > The command being run on Backup or Desktop is: > rsync -av -e 'ssh -i id_ecdsa' --exclude XA/ --exclude video2/ > root@10.5.5.1:/bulk/mux/media/xbn* . --progress > > > > > Things I did: > ------------- > Tried the still-installed F26 and F25 kernels > Desktop (F26) can rsync to and from Server(F26) at full speed. > All machines get 110MB/s when running iperf, all six ways. > The Sky2 nic in Backup is not spewing errors > iperf shows no dropped. > Swapped switchports between Desktop and Backup > Switch does not report errors > > Things to do: > ------------- > swap nics > upgrade a sacrificial F26 machine to F27 and try from that _______________________________________________ users mailing list -- users@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to users-leave@xxxxxxxxxxxxxxxxxxxxxxx
