This is a follow-up to my earlier email. I still haven't been able to figure out what the problem is, but if I turn off AFR between the servers, write performance improves to be about the same as NFS. I can send my config files again, but basically I comment out the afr sections and point my client directly to one of the servers by changing the "remote-host" option in the client vol file. I'm testing write performance with time sh -c "dd if=/dev/zero of=ddfile bs=8k count=20000 && sync" time sh -c "dd if=/dev/zero of=ddfile-gfs bs=8k count=200000 && sync" Any ideas? Thanks, Matt Matt M wrote: > Hi All, > > I'm new to gluster and have a basic test environment of three old PCs: > two servers and one client. I've currently got it configured to do AFR > on the two servers and HA on the client, according to this example: > http://www.gluster.org/docs/index.php/High-availability_storage_using_server-side_AFR > > > I'm trying to figure out why NFS seems significantly faster in my > (basic) tests. My config files and results are below. Any help is > greatly appreciated! > > server1 (garnet) is SUSE SLES 9(OES1), gluster 2.0.0rc7, FUSE 2.5.3, > 2.6.5-7.316-smp > server2 (or) is SUSE SLES 10, gluster 2.0.0rc7, FUSE 2.7.2, > 2.6.16.60-0.34-default > client1 (charon) is SUSE SLES 10, gluster 2.0.0rc7, FUSE 2.7.2, > 2.6.16.60-0.34-default > > ---- > RESULTS > all tests performed on the client -- /gfs is my glusterfs mount and /nfs > is the gluster FS shared from server1 via NFS: > > GFS - no performance translators > time find /gfs/users/1 -type f > 0.768u 1.460s 1:59.09 1.8% 0+0k 0+0io 0pf+0w > > GFS - w/readahead and writeback: > 0.784u 1.860s 1:59.62 2.2% 0+0k 0+0io 0pf+0w > > NFS > time find /nfs/users/1 -type f > 0.584u 3.796s 0:37.96 11.5% 0+0k 0+0io 0pf+0w > > NFS - after an umount/mount > time find /nfs/users/1 -type f > 0.556u 3.224s 0:40.57 9.2% 0+0k 0+0io 0pf+0w > > GFS - dd > Directory: /gfs/users > [charon: users]# time sh -c "dd if=/dev/zero of=ddfile bs=8k > count=2000000 && sync" > 2000000+0 records in > 2000000+0 records out > 16384000000 bytes (16 GB) copied, 7065.52 seconds, 2.3 MB/s > 1.488u 13.440s 1:57:45.64 0.2% 0+0k 0+0io 1pf+0w > > NFS - dd > (unmount NFS volume, remount it) > Directory: /nfs/users > [charon: users]# time sh -c "dd if=/dev/zero of=ddfile bs=8k > count=2000000 && sync" > 2000000+0 records in > 2000000+0 records out > 16384000000 bytes (16 GB) copied, 1582.31 seconds, 10.4 MB/s > 2.640u 125.299s 26:22.70 8.0% 0+0k 0+0io 5pf+0w > ---- > > CONFIGS: > -- > server1 (garnet) > [garnet: users]# cat /etc/gluster/glusterfsd-ha-afr.vol > > # dataspace on garnet > volume gfs-ds > type storage/posix > option directory /export/home > end-volume > > # posix locks > volume gfs-ds-locks > type features/posix-locks > subvolumes gfs-ds > end-volume > > # dataspace on or > volume gfs-or-ds > type protocol/client > option transport-type tcp/client > option remote-host 152.xx.xx.xx > option remote-subvolume gfs-ds-locks > option transport-timeout 10 > end-volume > > # automatic file replication translator for dataspace > volume gfs-ds-afr > type cluster/afr > subvolumes gfs-ds-locks gfs-or-ds # local and remote dataspaces > end-volume > > # the actual volume to export > volume users > type performance/io-threads > option thread-count 8 > subvolumes gfs-ds-afr > end-volume > > # make the home volume available as a server share > volume server > type protocol/server > option transport-type tcp > subvolumes users > option auth.addr.gfs-ds-locks.allow 152.xx.xx.* > option auth.addr.users.allow 152.xx.xx.* > end-volume > > -- > server2 (or) > [or: gluster]# cat /etc/gluster/glusterfsd-ha-afr.vol > > # dataspace on or > volume gfs-ds > type storage/posix > option directory /export/home > end-volume > > # posix locks > volume gfs-ds-locks > type features/posix-locks > subvolumes gfs-ds > end-volume > > # dataspace on garent > volume gfs-garnet-ds > type protocol/client > option transport-type tcp/client > option remote-host 152.xx.xx.xx > option remote-subvolume gfs-ds-locks > option transport-timeout 10 > end-volume > > # automatic file replication translator for dataspace > volume gfs-ds-afr > type cluster/afr > subvolumes gfs-ds-locks gfs-garnet-ds # local and remote dataspaces > end-volume > > # the actual volume to export > volume users > type performance/io-threads > option thread-count 8 > subvolumes gfs-ds-afr > end-volume > > # make the users volume available as a server share > volume server > type protocol/server > option transport-type tcp > subvolumes users > option auth.addr.gfs-ds-locks.allow 152.xx.xx.* > option auth.addr.users.allow 152.xx.xx.* > end-volume > > -- > client1 (charon) > [root at charon:users]# cat /etc/gluster/glusterfs-ha.vol > > # the exported volume to mount > volume gluster > type protocol/client > option transport-type tcp/client > option remote-host gluster.example.com # RRDNS > option remote-subvolume users # exported volume > option transport-timeout 10 > end-volume > > # performance block for gluster > volume writeback > type performance/write-behind > option aggregate-size 131072 > subvolumes gluster > end-volume > > # performance block for gluster > volume readahead > type performance/read-ahead > option page-size 65536 > option page-count 16 > subvolumes writeback > end-volume > > volume ioc > type performance/io-cache > option cache-size 128MB > subvolumes readahead > end-volume > > Thanks! > -Matt > >
