Hi All I am testing the following scenario: A DRBD mirror between two servers, which heartbeat failover the drbd primary, gnbd export and ip address. I am trying to find potential bottlenecks, and have done the following tests. Network speed between the DRBD servers (A and B) --------------------------------------------------------- (A) dd if=/dev/zero bs=1G count=1 | nc 10.100.1.2 5001 (B) nc k -l 5001 | dd of=/dev/null (A) reports: 1+0 records in 1+0 records out 1073741824 bytes (1.1 GB) copied, 7.6384 seconds, 141 MB/s DRBD sync speed: ---------------------------------------------------------- dd if=/dev/zero bs=1G count=1 of=/dev/drbd0 oflag=sync 1+0 records in 1+0 records out 1073741824 bytes (1.1 GB) copied, 10.7832 seconds, 99.6 MB/s Network speed between GNBD export (A) and import (C) ----------------------------------------------------------- (C) dd if=/dev/zero bs=1G count=1 | nc nfs1 5001 (A) nc -k -l 5001 | dd of=/dev/null (C) reports: 1+0 records in 1+0 records out 1073741824 bytes (1.1 GB) copied, 10.4001 seconds, 103 MB/s Network speed between GNBD import (C) and export (A) ----------------------------------------------------------- (A) dd if=/dev/zero bs=1G count=1 | nc 10.200.3.10 5001 (C) nc -k -l 5001 | dd of=/dev/null (A) reports: 1+0 records in 1+0 records out 1073741824 bytes (1.1 GB) copied, 10.4001 seconds, 93 MB/s So I have established that writing to drbd directly is fast, and network speed is fast However, using gnbd as follows: on the drbd server: gnbd_serv -n /sbin/gnbd_export -c -e r0 -d /dev/drbd0 On the client: gnbd_import -i 10.200.3.3 I try the speed tests over the gnbd devices: Reading from GNBD: ------------------------------------------------------------ dd if=/dev/gnbd0 of=/dev/null bs=1G count=1 1+0 records in 1+0 records out 1073741824 bytes (1.1 GB) copied, 17.0842 seconds, 62.8 MB/s Writing to GNBD (no sync flag) ------------------------------------------------------------ dd if=/dev/zero of=/dev/gnbd0 bs=1G count=1 1+0 records in 1+0 records out 1073741824 bytes (1.1 GB) copied, 54.4142 seconds, 19.7 MB/s Writing to GNBD (sync flag) ------------------------------------------------------------ dd if=/dev/zero of=/dev/gnbd0 bs=1G count=1 oflag=sync 1+0 records in 1+0 records out 1073741824 bytes (1.1 GB) copied, 53.3085 seconds, 20.1 MB/s I am almost happy with the 62 Mb/s read speed, but the 20 MB/sec write speed seems a bit low, compared to the write rate to drbd, and the network speed. Can anyone give any hints for optimising the gnbd write speed (and read speed) Chris -- Linux-cluster mailing list Linux-cluster@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/linux-cluster
