Re: DRBD very slow....

[Ross Walker <rswwalker@xxxxxxxxx>]

On Jul 27, 2009, at 4:09 PM, Coert Waagmeester <lgroups@xxxxxxxxxxxxxxxxx> wrote:

On Mon, 2009-07-27 at 12:37 +0200, Coert Waagmeester wrote:

On Mon, 2009-07-27 at 12:02 +0200, Alexander Dalloz wrote:

On Mon, 2009-07-27 at 08:30 +0200, Coert Waagmeester wrote:

Hello Roman,

I am running drbd 8.2.6 (the standard centos version)

have you considered to test the drbd-8.3 packages?

http://bugs.centos.org/view.php?id=3598

http://dev.centos.org/centos/5/testing/{i386,x86_64}/RPMS/

Thank you very much for this tip! It was one very obvious place where I

did not look yet.

Would it be necessary to still recompile it for the TCP_NODELAY and

such?

I am just making sure, because

http://www.nabble.com/Huge-latency-issue-with-8.2.6-td18947965.html

makes it seem unnecessary.

Why do the repositories provide both DRBD 8.0.x and 8.2.6?

Here is a status update....
_______________________________________________________________________________
on both hosts I now run from the testing repository:
# rpm -qa | grep drbd
drbd83-8.3.1-5.el5.centos
kmod-drbd83-xen-8.3.1-4.el5.centos
_______________________________________________________________________________
Here is my config (slightly condensed):
-----------------------------------------------------
global {
 usage-count yes;
}
common {
 protocol C;
 syncer { rate 50M; }
 net {
#        allow-two-primaries; }
        sndbuf-size 0; }
#  disk {no-disk-flushes;
#        no-md-flushes; }
 startup { wfc-timeout 0 ; }
}
resource xenfilesrv {
 device    /dev/drbd1;
 disk      /dev/vg0/xenfilesrv;
 meta-disk internal;

 on baldur.mydomain.local {
   address   10.99.99.1:7788;
 }
 on thor.mydomain.local {
   address   10.99.99.2:7788;
 }
}
resource xenfilesrvdata {
 device    /dev/drbd2;
 disk      /dev/vg0/xenfilesrvdata;
 meta-disk internal;

 on baldur.mydomain.local {
   address   10.99.99.1:7789;
 }
 on thor.mydomain.local {
   address   10.99.99.2:7789;
 }
}
_______________________________________________________________________________

xenfilesrv is a xen domU
in this domU i ran a dd with oflag direct:
---------------------------------------------------------
# dd if=/dev/zero of=1gig.file bs=1M count=1000 oflag=direct
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB) copied, 147.997 seconds, 7.1 MB/s

Just before I ran the dd this popped up in the secondary hosts syslog:
----------------------------------------------------------------------
Jul 27 21:51:42 thor kernel: drbd2: Method to ensure write ordering:
flush
Jul 27 21:51:42 thor kernel: drbd1: Method to ensure write ordering:
flush


_______________________________________________________________________________

What more can I try?

To be quite honest, I have no idea what to do with/ where to find the
TCP_NODELAY socket options......

Use drbd option to disable flush/sync, but understand that during a power failure or system crash data will not be consistent on disk and you will need to sync the storage from the other server.

-Ross

_______________________________________________
CentOS mailing list
CentOS@xxxxxxxxxx
http://lists.centos.org/mailman/listinfo/centos