I have a very strange issue with 2.0.1. I have 2 systems. On each system there is server AND client running. The 2 servers are using serverside afr/replicate and the client on each is connected to a single brick/volume exported on his local server. The client does not know anything about the other server. Now when I benchmark and look what get's over the wire, then I can not get more then +/- 14MB/s. No matter what performance translators I enable/disable. The speed is always around 14MB/s. Now if I put NUFA on top of replicate, things change. I get faster transfer but the file written does not get transferred to the other server (I see NO network traffic in that regard). But if I go on the other server and do a simple "ls", then network traffic goes up to +/- 50MB/s and the file shows up on the other server/client. That sound sounds to me normal (well... that's probably NUFA responsible in favoring the local disc). However... after the "ls" command on the server/client where the file was transferred to, the other server/client crashes with the following log: +------------------------------------------------------------------------------+ [2009-06-05 22:31:48] N [afr.c:2190:notify] gfs-srv-ds-replicate: Subvolume 'gfs-srv-ds-locks' came back up; going online. [2009-06-05 22:31:48] N [afr.c:2190:notify] gfs-srv-ds-replicate: Subvolume 'gfs-srv-ds-locks' came back up; going online. [2009-06-05 22:31:48] N [afr.c:2190:notify] gfs-srv-ds-replicate: Subvolume 'gfs-srv-ds-locks' came back up; going online. [2009-06-05 22:31:48] N [afr.c:2190:notify] gfs-srv-ds-replicate: Subvolume 'gfs-srv-ds-locks' came back up; going online. [2009-06-05 22:31:48] N [glusterfsd.c:1152:main] glusterfs: Successfully started [2009-06-05 22:31:48] N [client-protocol.c:5557:client_setvolume_cbk] gfs-srv-ds-remote: Connected to 192.168.0.77:6997, attached to remote volume 'gfs-srv-ds-locks'. [2009-06-05 22:31:48] N [client-protocol.c:5557:client_setvolume_cbk] gfs-srv-ds-remote: Connected to 192.168.0.77:6997, attached to remote volume 'gfs-srv-ds-locks'. [2009-06-05 22:31:51] N [server-protocol.c:7035:mop_setvolume] gfs-srv-ds-server: accepted client from 192.168.0.77:1021 [2009-06-05 22:31:51] N [server-protocol.c:7035:mop_setvolume] gfs-srv-ds-server: accepted client from 127.0.0.1:1023 [2009-06-05 22:31:51] N [server-protocol.c:7035:mop_setvolume] gfs-srv-ds-server: accepted client from 127.0.0.1:1022 [2009-06-05 22:31:51] N [server-protocol.c:7035:mop_setvolume] gfs-srv-ds-server: accepted client from 192.168.0.77:1020 pending frames: frame : type(1) op(LOOKUP) patchset: 5c1d9108c1529a1155963cb1911f8870a674ab5b signal received: 11 configuration details:argp 1 backtrace 1 dlfcn 1 fdatasync 1 libpthread 1 llistxattr 1 setfsid 1 spinlock 1 xattr.h 1 st_atim.tv_nsec 1 package-string: glusterfs 2.0.1 [0xffffe400] /usr/lib/glusterfs/2.0.1/xlator/protocol/client.so(client_lookup+0x96)[0xb75573fb] /usr/lib/glusterfs/2.0.1/xlator/cluster/replicate.so(afr_lookup+0x22f)[0xb7517a05] /usr/lib/glusterfs/2.0.1/xlator/cluster/nufa.so(nufa_lookup+0x3ea)[0xb7503fa9] /usr/lib/glusterfs/2.0.1/xlator/performance/io-threads.so(iot_lookup_wrapper+0xa5)[0xb74e8415] /usr/lib/libglusterfs.so.0(call_resume+0x344)[0xb7f47e01] /usr/lib/glusterfs/2.0.1/xlator/performance/io-threads.so(iot_worker_unordered+0x20)[0xb74e5895] /lib/libpthread.so.0[0xb7f154cf] /lib/libc.so.6(clone+0x5e)[0xb7e9b27e] --------- Now my questions: 1) Is this issue known? I can reproduce that error and therefore I could send more info if needed. 2) Why is Server 1 AFR/Replicate <-> Server 2 AFR/Replicate so slow? Just 14MB/s on GigE seems slow to me. Writing directly to the local disk (without GlusterFS) delivers +/- 57MB/s. Going over NFSv4 delivers +/- 45MB/s. Going over SSH delivers +/- 33MB/s. Transfer from Server 1 to Server 2 with the GlusterFS log delivers +/- 45MB/s. Just pure Server to Server with AFR/Replicate only delivers 14MB/s. Why? // Steve -- GRATIS für alle GMX-Mitglieder: Die maxdome Movie-FLAT! Jetzt freischalten unter http://portal.gmx.net/de/go/maxdome01
