Hello! After upgrade to version 2.0, now using 2.0.1, I'm experiencing problems with glusterfs stability. I'm running 2 node setup with cliet side afr, and glusterfsd also is running on same servers. Time to time glusterfs just hangs, i can reproduce this running iozone benchmarking tool. I'm using patched Fuse, but same result is with unpatched. ================================================================================ Version : glusterfs 2.0.1 built on May 27 2009 16:04:01 TLA Revision : 5c1d9108c1529a1155963cb1911f8870a674ab5b Starting Time: 2009-05-27 16:38:20 Command line : /usr/sbin/glusterfsd --volfile=/etc/glusterfs/glusterfs-server.vol --pid-file=/var/run/glusterfsd.pid --log-file=/var/log/glusterfsd.log PID : 31971 System name : Linux Nodename : weeber.st-inst.lv Kernel Release : 2.6.28-hardened-r7 Hardware Identifier: i686 Given volfile: +------------------------------------------------------------------------------+ 1: # file: /etc/glusterfs/glusterfs-server.vol 2: volume posix 3: type storage/posix 4: option directory /home/export 5: end-volume 6: 7: volume locks 8: type features/locks 9: option mandatory-locks on 10: subvolumes posix 11: end-volume 12: 13: volume brick 14: type performance/io-threads 15: option autoscaling on 16: subvolumes locks 17: end-volume 18: 19: volume server 20: type protocol/server 21: option transport-type tcp 22: option auth.addr.brick.allow 127.0.0.1,192.168.1.* 23: subvolumes brick 24: end-volume +------------------------------------------------------------------------------+ [2009-05-27 16:38:20] N [glusterfsd.c:1152:main] glusterfs: Successfully started [2009-05-27 16:38:33] N [server-protocol.c:7035:mop_setvolume] server: accepted client from 192.168.1.233:1021 [2009-05-27 16:38:33] N [server-protocol.c:7035:mop_setvolume] server: accepted client from 192.168.1.233:1020 [2009-05-27 16:38:46] N [server-protocol.c:7035:mop_setvolume] server: accepted client from 192.168.1.252:1021 [2009-05-27 16:38:46] N [server-protocol.c:7035:mop_setvolume] server: accepted client from 192.168.1.252:1020 ================================================================================ Version : glusterfs 2.0.1 built on May 27 2009 16:04:01 TLA Revision : 5c1d9108c1529a1155963cb1911f8870a674ab5b Starting Time: 2009-05-27 16:38:46 Command line : /usr/sbin/glusterfs -N -f /etc/glusterfs/glusterfs-client.vol /mnt/gluster PID : 32161 System name : Linux Nodename : weeber.st-inst.lv Kernel Release : 2.6.28-hardened-r7 Hardware Identifier: i686 Given volfile: +------------------------------------------------------------------------------+ 1: volume xeon 2: type protocol/client 3: option transport-type tcp 4: option remote-host 192.168.1.233 5: option remote-subvolume brick 6: end-volume 7: 8: volume weeber 9: type protocol/client 10: option transport-type tcp 11: option remote-host 192.168.1.252 12: option remote-subvolume brick 13: end-volume 14: 15: volume replicate 16: type cluster/replicate 17: subvolumes xeon weeber 18: end-volume 20: volume readahead 21: type performance/read-ahead 22: option page-size 128kB 23: option page-count 16 24: option force-atime-update off 25: subvolumes replicate 26: end-volume 27: 28: volume writebehind 29: type performance/write-behind 30: option aggregate-size 1MB 31: option window-size 3MB 32: option flush-behind on 33: option enable-O_SYNC on 34: subvolumes readahead 35: end-volume 36: 37: volume iothreads 38: type performance/io-threads 39: option autoscaling on 40: subvolumes writebehind 41: end-volume 42: 43: 44: 45: #volume bricks 46: #type cluster/distribute 47: #option lookup-unhashed yes 48: #option min-free-disk 20% 49: # subvolumes weeber xeon 50: #end-volume +------------------------------------------------------------------------------+ [2009-05-27 16:38:46] W [xlator.c:555:validate_xlator_volume_options] writebehind: option 'window-size' is deprecated, preferred is 'cache-size', continuing with correction [2009-05-27 16:38:46] W [glusterfsd.c:455:_log_if_option_is_invalid] writebehind: option 'aggregate-size' is not recognized [2009-05-27 16:38:46] W [glusterfsd.c:455:_log_if_option_is_invalid] readahead: option 'page-size' is not recognized [2009-05-27 16:38:46] N [glusterfsd.c:1152:main] glusterfs: Successfully started [2009-05-27 16:38:46] N [client-protocol.c:5557:client_setvolume_cbk] xeon: Connected to 192.168.1.233:6996, attached to remote volume 'brick'. [2009-05-27 16:38:46] N [afr.c:2190:notify] replicate: Subvolume 'xeon' came back up; going online. [2009-05-27 16:38:46] N [client-protocol.c:5557:client_setvolume_cbk] xeon: Connected to 192.168.1.233:6996, attached to remote volume 'brick'. [2009-05-27 16:38:46] N [afr.c:2190:notify] replicate: Subvolume 'xeon' came back up; going online. [2009-05-27 16:38:46] N [client-protocol.c:5557:client_setvolume_cbk] weeber: Connected to 192.168.1.252:6996, attached to remote volume 'brick'. [2009-05-27 18:46:02] E [client-protocol.c:292:call_bail] weeber: bailing out frame LOOKUP(32) frame sent = 2009-05-27 18:16:01. frame-timeout = 1800 [2009-05-27 19:16:09] E [client-protocol.c:292:call_bail] weeber: bailing out frame LOOKUP(32) frame sent = 2009-05-27 18:46:02. frame-timeout = 1800 [2009-05-27 19:46:18] E [client-protocol.c:292:call_bail] weeber: bailing out frame OPEN(12) frame sent = 2009-05-27 19:16:09. frame-timeout = 1800 [2009-05-27 20:16:25] E [client-protocol.c:292:call_bail] weeber: bailing out frame LOOKUP(32) frame sent = 2009-05-27 19:46:18. frame-timeout = 1800 [2009-05-27 20:46:34] E [client-protocol.c:292:call_bail] weeber: bailing out frame LOOKUP(32) frame sent = 2009-05-27 20:16:25. frame-timeout = 1800 [2009-05-27 21:16:41] E [client-protocol.c:292:call_bail] weeber: bailing out frame OPEN(12) frame sent = 2009-05-27 20:46:34. frame-timeout = 1800 [2009-05-27 21:47:00] E [client-protocol.c:292:call_bail] weeber: bailing out frame LOOKUP(32) frame sent = 2009-05-27 21:16:53. frame-timeout = 1800 [2009-05-27 22:17:07] E [client-protocol.c:292:call_bail] weeber: bailing out frame LOOKUP(32) frame sent = 2009-05-27 21:47:00. frame-timeout = 1800 [2009-05-27 22:47:15] E [client-protocol.c:292:call_bail] weeber: bailing out frame OPENDIR(21) frame sent = 2009-05-27 22:17:07. frame-timeout = 1800 [2009-05-27 23:17:23] E [client-protocol.c:292:call_bail] weeber: bailing out frame LOOKUP(32) frame sent = 2009-05-27 22:47:15. frame-timeout = 1800 [2009-05-27 23:47:31] E [client-protocol.c:292:call_bail] weeber: bailing out frame OPEN(12) frame sent = 2009-05-27 23:17:23. frame-timeout = 1800 [2009-05-28 00:17:39] E [client-protocol.c:292:call_bail] weeber: bailing out frame LOOKUP(32) frame sent = 2009-05-27 23:47:32. frame-timeout = 1800 [2009-05-28 00:47:47] E [client-protocol.c:292:call_bail] weeber: bailing out frame LOOKUP(32) frame sent = 2009-05-28 00:17:39. frame-timeout = 1800 [2009-05-28 01:17:55] E [client-protocol.c:292:call_bail] weeber: bailing out frame OPENDIR(21) frame sent = 2009-05-28 00:47:47. frame-timeout = 1800 [2009-05-28 01:48:03] E [client-protocol.c:292:call_bail] weeber: bailing out frame LOOKUP(32) frame sent = 2009-05-28 01:17:55. frame-timeout = 1800 [2009-05-28 02:18:11] E [client-protocol.c:292:call_bail] weeber: bailing out frame OPEN(12) frame sent = 2009-05-28 01:48:03. frame-timeout = 1800 [2009-05-28 02:48:29] E [client-protocol.c:292:call_bail] weeber: bailing out frame LOOKUP(32) frame sent = 2009-05-28 02:18:24. frame-timeout = 1800 [2009-05-28 03:18:37] E [client-protocol.c:292:call_bail] weeber: bailing out frame LOOKUP(32) frame sent = 2009-05-28 02:48:29. frame-timeout = 1800 [2009-05-28 03:48:45] E [client-protocol.c:292:call_bail] weeber: bailing out frame LOOKUP(32) frame sent = 2009-05-28 03:18:37. frame-timeout = 1800 [2009-05-28 04:18:53] E [client-protocol.c:292:call_bail] weeber: bailing out frame XATTROP(40) frame sent = 2009-05-28 03:48:45. frame-timeout = 1800 [2009-05-28 04:49:01] E [client-protocol.c:292:call_bail] weeber: bailing out frame LOOKUP(32) frame sent = 2009-05-28 04:18:53. frame-timeout = 1800 [2009-05-28 05:19:09] E [client-protocol.c:292:call_bail] weeber: bailing out frame OPENDIR(21) frame sent = 2009-05-28 04:49:01. frame-timeout = 1800 [2009-05-28 05:49:17] E [client-protocol.c:292:call_bail] weeber: bailing out frame LOOKUP(32) frame sent = 2009-05-28 05:19:09. frame-timeout = 1800 [2009-05-28 06:19:25] E [client-protocol.c:292:call_bail] weeber: bailing out frame LOOKUP(32) frame sent = 2009-05-28 05:49:17. frame-timeout = 1800 [2009-05-28 06:49:33] E [client-protocol.c:292:call_bail] weeber: bailing out frame XATTROP(40) frame sent = 2009-05-28 06:19:25. frame-timeout = 1800 [2009-05-28 07:19:40] E [client-protocol.c:292:call_bail] weeber: bailing out frame LOOKUP(32) frame sent = 2009-05-28 06:49:33. frame-timeout = 1800 [2009-05-28 07:49:48] E [client-protocol.c:292:call_bail] weeber: bailing out frame LOOKUP(32) frame sent = 2009-05-28 07:19:40. frame-timeout = 1800 [2009-05-28 08:19:56] E [client-protocol.c:292:call_bail] weeber: bailing out frame LOOKUP(32) frame sent = 2009-05-28 07:49:48. frame-timeout = 1800 -------------- next part -------------- A non-text attachment was scrubbed... Name: maris.vcf Type: text/x-vcard Size: 206 bytes Desc: not available URL: <http://zresearch.com/pipermail/gluster-users/attachments/20090528/b863fae2/attachment-0001.vcf>
