Hi, when our two-node distributed backup storage cluster, running on GlusterFS 3.0.2 (Ubuntu Lucid packages), started running out of space, I added a third node and tried rebalancing them. The scale-n-defrag script didn't work, though. I always got a "stale NFS handle". So I decided to upgrade and installed the 3.2.4 packages from semiosis' Launchpad PPA, followed the upgrade guide for 3.0 -> 3.2 and then started "gluster volume rebalance". After two days, the three nodes are now about equally full, so the rebalancing worked. Unfortunately, it seems that it also broke the cluster filesystem. We get a lot of "permission denied" errors although file system permissions are correct. The log contains a lot of these entries: [2011-10-17 10:11:48.979801] I [client3_1-fops.c:2228:client3_1_lookup_cbk] 0-glfs-client-2: remote operation failed: Permission denied [2011-10-17 10:11:48.979892] I [dht-common.c:478:dht_revalidate_cbk] 0-glfs-dht: subvolume glfs-client-2 for /bacula/mail02 returned -1 (Permission denied) [2011-10-17 10:11:48.980216] I [dht-layout.c:192:dht_layout_search] 0-glfs-dht: no subvolume for hash (value) = 1486987642 [2011-10-17 10:11:48.981101] I [client3_1-fops.c:2228:client3_1_lookup_cbk] 0-glfs-client-2: remote operation failed: Permission denied [2011-10-17 10:11:48.981581] I [dht-layout.c:192:dht_layout_search] 0-glfs-dht: no subvolume for hash (value) = 1486987642 [2011-10-17 10:11:48.982468] I [client3_1-fops.c:2228:client3_1_lookup_cbk] 0-glfs-client-2: remote operation failed: Permission denied How can I repair this? Best regards, Jochen