Hi Jeff. No, because the client "bacula-fd" is reading from the local filesystem - here CephFS and sending it over the network to the server with the tape-libraries attached to it. Thus "ideal" receive == send - which is also the pattern I see when using larger files (multiple MB). Is the per-file overhead many KB? Jesper On Mon, Apr 6, 2020 at 1:45 PM Jeff Layton <jlayton@xxxxxxxxxx> wrote: > > On Mon, 2020-04-06 at 10:04 +0200, Jesper Krogh wrote: > > This is a CephFS client - its only purpose is to run the "filedaemon" of bacula > > and transport data to the tape-library from CephFS - below 2 threads is > > essentially doing something equivalent to > > > > find /cephfs/ -type f | xargs cat | nc server > > > > 2 threads only, load exploding and the "net read vs net write" has > > more than 100x difference. > > > > Makes sense. You're basically just reading in all of the data on this > cephfs, so the receive is going to be much larger than the send. > > > Can anyone explain this as "normal" behaviour? > > Server is a VM with 16 "vCPU" and 16GB memory running libvirt/qemu > > > > jk@wombat:~$ w > > 07:50:33 up 11:25, 1 user, load average: 206.43, 76.23, 50.58 > > USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT > > jk pts/0 10.194.133.42 06:54 0.00s 0.05s 0.00s w > > jk@wombat:~$ dstat -ar > > --total-cpu-usage-- -dsk/total- -net/total- ---paging-- ---system-- --io/total- > > usr sys idl wai stl| read writ| recv send| in out | int csw | read writ > > 0 0 98 1 0| 14k 34k| 0 0 | 3B 27B| 481 294 |0.55 0.73 > > 1 1 0 98 0| 0 0 | 60M 220k| 0 0 |6402 6182 | 0 0 > > 0 1 0 98 0| 0 0 | 69M 255k| 0 0 |7305 4339 | 0 0 > > 1 2 0 98 0| 0 0 | 76M 282k| 0 0 |7914 4886 | 0 0 > > 1 1 0 99 0| 0 0 | 70M 260k| 0 0 |7293 4444 | 0 0 > > 1 1 0 98 0| 0 0 | 80M 278k| 0 0 |8018 4931 | 0 0 > > 0 1 0 98 0| 0 0 | 60M 221k| 0 0 |6435 5951 | 0 0 > > 0 1 0 99 0| 0 0 | 59M 211k| 0 0 |6163 3584 | 0 0 > > 0 1 0 98 0| 0 0 | 64M 323k| 0 0 |6653 3881 | 0 0 > > 1 0 0 99 0| 0 0 | 61M 243k| 0 0 |6822 4401 | 0 0 > > 0 1 0 99 0| 0 0 | 55M 205k| 0 0 |5975 3518 | 0 0 > > 1 1 0 98 0| 0 0 | 68M 242k| 0 0 |7094 6544 | 0 0 > > 0 1 0 99 0| 0 0 | 58M 230k| 0 0 |6639 4178 | 0 0 > > 1 2 0 98 0| 0 0 | 61M 243k| 0 0 |7117 4477 | 0 0 > > 0 1 0 99 0| 0 0 | 61M 228k| 0 0 |6500 4078 | 0 0 > > 0 1 0 99 0| 0 0 | 65M 234k| 0 0 |6595 3914 | 0 0 > > 0 1 0 98 0| 0 0 | 64M 219k| 0 0 |6507 5755 | 0 0 > > 1 1 0 99 0| 0 0 | 64M 233k| 0 0 |6869 4153 | 0 0 > > 1 2 0 98 0| 0 0 | 63M 232k| 0 0 |6632 3907 | > > 0 0 ^C > > Load average is high, but it looks like it's all just waiting on I/O. > > > jk@wombat:~$ w > > 07:50:56 up 11:25, 1 user, load average: 221.35, 88.07, 55.02 > > USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT > > jk pts/0 10.194.133.42 06:54 0.00s 0.05s 0.00s w > > jk@wombat:~$ > > > -- > Jeff Layton <jlayton@xxxxxxxxxx> >
