hi everyone, i'm afraid to bring bad news. after applying patch 650, the system seemed to work smoothly for a whole under a lot of work. I moved to patch 651 as you suggested, and it didn't last more than 3 hours :( the thing is... server didn't hang, but filesystem isn't accessible at all, not even an "ls" is possible from any of the clients. this are the node logs: ************* nothing logged about this matter!!! ************ and these are the client logs: ************* 2008-02-08 14:56:59 E [unify.c:260:unify_lookup_cbk] ultim: Revalidate failed for /dummy/maildirsize 2008-02-08 14:56:59 E [fuse-bridge.c:431:fuse_entry_cbk] glusterfs-fuse: 202928: /dummy/maildirsize => -1 (2) 2008-02-08 14:56:59 E [afr.c:2398:afr_selfheal_getxattr_cbk] grup2: (path=/dummy/maildirsize child=espai4) op_ret=-1 op_errno=2 2008-02-08 14:56:59 E [afr.c:2398:afr_selfheal_getxattr_cbk] grup2: (path=/dummy/maildirsize child=espai3) op_ret=-1 op_errno=2 2008-02-08 14:56:59 E [afr.c:1564:afr_open_cbk] grup2: (path=/dummy/maildirsize child=espai4) op_ret=-1 op_errno=2 2008-02-08 14:56:59 E [afr.c:1564:afr_open_cbk] grup2: (path=/dummy/maildirsize child=espai3) op_ret=-1 op_errno=2 2008-02-08 14:56:59 E [unify.c:802:unify_open_cbk] ultim: Open success on namespace, failed on child node 2008-02-08 14:56:59 E [fuse-bridge.c:670:fuse_fd_cbk] glusterfs-fuse: 203092: /dummy/maildirsize => -1 (2) 2008-02-08 14:56:59 E [afr.c:2398:afr_selfheal_getxattr_cbk] grup2: (path=/dummy/maildirsize child=espai4) op_ret=-1 op_errno=2 2008-02-08 14:56:59 E [afr.c:2398:afr_selfheal_getxattr_cbk] grup2: (path=/dummy/maildirsize child=espai3) op_ret=-1 op_errno=2 2008-02-08 14:56:59 E [afr.c:1564:afr_open_cbk] grup2: (path=/dummy/maildirsize child=espai4) op_ret=-1 op_errno=2 2008-02-08 14:56:59 E [afr.c:1564:afr_open_cbk] grup2: (path=/dummy/maildirsize child=espai3) op_ret=-1 op_errno=2 2008-02-08 14:56:59 E [unify.c:802:unify_open_cbk] ultim: Open success on namespace, failed on child node 2008-02-08 14:56:59 E [fuse-bridge.c:670:fuse_fd_cbk] glusterfs-fuse: 203093: /dummy/maildirsize => -1 (2) 2008-02-08 14:56:59 E [afr.c:2398:afr_selfheal_getxattr_cbk] grup2: (path=/dummy/maildirsize child=espai3) op_ret=-1 op_errno=2 2008-02-08 14:56:59 E [afr.c:2398:afr_selfheal_getxattr_cbk] grup2: (path=/dummy/maildirsize child=espai4) op_ret=-1 op_errno=2 2008-02-08 14:56:59 E [afr.c:1564:afr_open_cbk] grup2: (path=/dummy/maildirsize child=espai3) op_ret=-1 op_errno=2 2008-02-08 14:56:59 E [afr.c:1564:afr_open_cbk] grup2: (path=/dummy/maildirsize child=espai4) op_ret=-1 op_errno=2 2008-02-08 14:56:59 E [unify.c:802:unify_open_cbk] ultim: Open success on namespace, failed on child node 2008-02-08 14:56:59 E [fuse-bridge.c:670:fuse_fd_cbk] glusterfs-fuse: 203094: /dummy/maildirsize => -1 (2) 2008-02-08 14:57:00 E [unify.c:260:unify_lookup_cbk] ultim: Revalidate failed for /dummy/maildirsize 2008-02-08 14:57:00 E [fuse-bridge.c:431:fuse_entry_cbk] glusterfs-fuse: 203247: /dummy/maildirsize => -1 (2) 2008-02-08 17:33:05 E [afr.c:2398:afr_selfheal_getxattr_cbk] nm: (path=/dummy/maildirsize child=namespace2) op_ret=-1 op_errno=2 2008-02-08 17:33:05 E [afr.c:2398:afr_selfheal_getxattr_cbk] nm: (path=/dummy/maildirsize child=namespace1) op_ret=-1 op_errno=2 2008-02-08 17:33:05 E [afr.c:2398:afr_selfheal_getxattr_cbk] grup2: (path=/dummy/maildirsize child=espai4) op_ret=-1 op_errno=2 2008-02-08 17:33:05 E [afr.c:2398:afr_selfheal_getxattr_cbk] grup2: (path=/dummy/maildirsize child=espai3) op_ret=-1 op_errno=2 2008-02-08 17:33:05 E [afr.c:1564:afr_open_cbk] nm: (path=/dummy/maildirsize child=namespace2) op_ret=-1 op_errno=2 2008-02-08 17:33:05 E [afr.c:1564:afr_open_cbk] nm: (path=/dummy/maildirsize child=namespace1) op_ret=-1 op_errno=2 2008-02-08 17:33:05 E [afr.c:1564:afr_open_cbk] grup2: (path=/dummy/maildirsize child=espai4) op_ret=-1 op_errno=2 2008-02-08 17:33:05 E [afr.c:1564:afr_open_cbk] grup2: (path=/dummy/maildirsize child=espai3) op_ret=-1 op_errno=2 2008-02-08 17:33:05 E [fuse-bridge.c:670:fuse_fd_cbk] glusterfs-fuse: 398639: /dummy/maildirsize => -1 (2) 2008-02-08 17:49:46 E [unify.c:260:unify_lookup_cbk] ultim: Revalidate failed for /dummy/maildirsize 2008-02-08 17:49:46 E [fuse-bridge.c:431:fuse_entry_cbk] glusterfs-fuse: 517247: /dummy/maildirsize => -1 (2) 2008-02-08 18:03:38 E [afr.c:1564:afr_open_cbk] grup3: (path=/dummy/maildirsize child=espai6) op_ret=-1 op_errno=2 2008-02-08 18:03:38 E [afr.c:1564:afr_open_cbk] grup3: (path=/dummy/maildirsize child=espai5) op_ret=-1 op_errno=2 2008-02-08 18:03:38 E [unify.c:802:unify_open_cbk] ultim: Open success on namespace, failed on child node 2008-02-08 18:03:38 E [fuse-bridge.c:670:fuse_fd_cbk] glusterfs-fuse: 641123: /dummy/maildirsize => -1 (2) 2008-02-08 18:30:33 E [afr.c:1564:afr_open_cbk] grup2: (path=/dummy/maildirsize child=espai3) op_ret=-1 op_errno=2 2008-02-08 18:30:33 E [afr.c:1564:afr_open_cbk] grup2: (path=/dummy/maildirsize child=espai4) op_ret=-1 op_errno=2 2008-02-08 18:30:33 E [afr.c:1564:afr_open_cbk] nm: (path=/dummy/maildirsize child=namespace2) op_ret=-1 op_errno=2 2008-02-08 18:30:33 E [afr.c:1564:afr_open_cbk] nm: (path=/dummy/maildirsize child=namespace1) op_ret=-1 op_errno=2 2008-02-08 18:30:33 E [fuse-bridge.c:670:fuse_fd_cbk] glusterfs-fuse: 894293: /dummy/maildirsize => -1 (2) 2008-02-08 18:38:09 E [afr.c:1564:afr_open_cbk] nm: (path=/dummy/maildirsize child=namespace2) op_ret=-1 op_errno=2 2008-02-08 18:38:09 E [afr.c:1564:afr_open_cbk] nm: (path=/dummy/maildirsize child=namespace1) op_ret=-1 op_errno=2 2008-02-08 18:38:09 E [unify.c:794:unify_open_cbk] ultim: Open success on child node, failed on namespace 2008-02-08 18:38:09 E [fuse-bridge.c:670:fuse_fd_cbk] glusterfs-fuse: 969868: /dummy/maildirsize => -1 (2) 2008-02-08 19:26:58 E [unify.c:260:unify_lookup_cbk] ultim: Revalidate failed for /dummy/maildirsize 2008-02-08 19:26:58 E [fuse-bridge.c:431:fuse_entry_cbk] glusterfs-fuse: 1302224: /dummy/maildirsize => -1 (2) 2008-02-08 19:40:03 E [afr.c:1564:afr_open_cbk] nm: (path=/dummy/maildirsize child=namespace2) op_ret=-1 op_errno=2 2008-02-08 19:40:03 E [afr.c:1564:afr_open_cbk] nm: (path=/dummy/maildirsize child=namespace1) op_ret=-1 op_errno=2 2008-02-08 19:40:03 E [unify.c:794:unify_open_cbk] ultim: Open success on child node, failed on namespace ************* i don't know if that makes any sense, but here you've got come cats dovecots: cat /mnt/fusectl/1/waiting >> 6 postfixs: cat /mnt/fusectl/1/waiting >> 14 these are values at the very moment everything hanged
