If I am not mistaken, you have a single server for glusterfs, and this is mirrored to a second ( non-glusterFS ) server using DRBD. If you have only a single server to export data from, why use Glusterfs ? Also, we don't officially support DRBD replication with glusterFS backend. Maybe you can consider GlusterFS replication across the two servers ? ----- Original Message ----- From: "Ran" <smtp.test61 at gmail.com> To: "Tejas N. Bhise" <tejas at gluster.com> Sent: Thursday, May 20, 2010 6:30:21 PM Subject: Re: Big I/O or gluster proccess problem Tejas hi , The 2 servers DRBD pair with HA so gluster actualy has 1 server that export 1 H.D ( 1 TB) this H.D is DRBD'd to the other server also this 1 TB hd is raid 1 with linux raid (i know its not optimal but rebust) in this setup if 1 servers go down the other continue , DRBD is more rebust then gluster replication specialy for VPS's etc.. I didnt check iowait but the load of the server is about 5 while the CPU's are 10-50% only so that says it all(there are IO waits) I was thinking of breaking the raid 1 seens this H.D allready has full mirror with DRBD( to server 2) but im not sure it will resolve this problem seens with NFS its not the same , it slows thigs down but not to not functional . client vol file 192.168.0.9 is the HA IP of this pair iv tested also with plain config(no writebehind etc) <> # file: /etc/glusterfs/glusterfs.vol volume storage1-2 type protocol/client option transport-type tcp option remote-host 192.168.0.9 option remote-subvolume b1 option ping-timeout 120 option username .......... option password .......... end-volume volume cluster type cluster/distribute option lookup-unhashed yes subvolumes storage1-2 end-volume #volume writebehind # type performance/write-behind # option cache-size 3MB # subvolumes cluster #end-volume #volume readahead # type performance/read-ahead # option page-count 4 # subvolumes writebehind #end-volume volume iothreads type performance/io-threads option thread-count 4 subvolumes cluster end-volume volume io-cache type performance/io-cache option cache-size 128MB option page-size 256KB #128KB is default option option force-revalidate-timeout 10 # default is 1 subvolumes iothreads end-volume volume writebehind type performance/write-behind option aggregate-size 512KB # default is 0bytes option flush-behind on # default is 'off' subvolumes io-cache end-volume <> server vol file <> # file: /etc/glusterfs/glusterfs-server.vol volume posix type storage/posix option directory /data/gluster # option o-direct enable option background-unlink yes # option span-devices 8 end-volume volume locks type features/locks subvolumes posix end-volume volume b1 type performance/io-threads option thread-count 8 subvolumes locks end-volume volume server type protocol/server option transport.socket.nodelay on option transport-type tcp # option auth.addr.b1.allow * option auth.login.b1.allow .......... option auth.login.gluster.password ................ subvolumes b1 end-volume <> 2010/5/20 Tejas N. Bhise < tejas at gluster.com > Ran, Can you please elaborate on - "2 servers in distrebute mod , each has 1 TB brick that replicate to each other using DRBD" Also, how many drives do you have and what does iowait look like when you write a big file ? Tell us more about the configs of your servers, share the volume files. Regards, Tejas. ----- Original Message ----- From: "Ran" < smtp.test61 at gmail.com > To: Gluster-users at gluster.org Sent: Thursday, May 20, 2010 4:49:52 PM Subject: Big I/O or gluster proccess problem Hi all , Our problem is simple but quite critical i posted few mounts ago regarding that issue and there was a good responses but not a fix What happen is that gluster stack when there is a big wrtite to it . For example time dd if=/dev/zero of=file bs=10240 count=1000000 OR mv 20gig_file.img into gluster mount . When that happen the all storage freezes for the entire proccess , mails , few vps's , simple dir etc.. Our setup is quite simple at this point , 2 servers in distrebute mod , each has 1 TB brick that replicate to each other using DRBD . iv monitored closly everting on this big writes and noticed that its not mem , proc , network problem . Iv also checked the same setup with simple NFS and its not happening there . Anyone has any idea on how to fix this , if you cant write big files into gluster without making the storage unfunctional then you cant realy do anything . Please advise , _______________________________________________ Gluster-users mailing list Gluster-users at gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
