Hi , Im running gluster 3.1.4 . [glusterfs 3.1.4 built on May 5 2011 06:52:24 Repository revision: v3.1.4] Assumed test scenario: [Might not be correct but the test python script gives a definite repro] 1. create few files , write few thousands of lines of data in these files. 2. now create another file in the same dir and Rename or move the new file with the old file name. 3. Try these for few number of files. 4 After few iterations opening these files with any application will give "Permission denied." In my test i was using python apis - os.rename and file open apis. This is very easy to reproduce with the attached python script *Note*: before running the script create the required directories mentioned in the script as well as create each input directories with atleast 1500 files. The issue seems to be related to DHT. When I delete the link files with sticky bit ------T from the backend after the application returns "permission denied", the file becomes accessible. logs: [2011-05-20 10:02:41.821605] I [client-handshake.c:1014:select_server_supported_programs] 1-172.26.98.57-1: Using Program GlusterFS-3.1.0, Num (1298437), Version (310) [2011-05-20 10:02:41.821696] I [client-handshake.c:852:client_setvolume_cbk] 1-172.26.98.56-1: Connected to 172.26.98.56:6996, attached to remote volume 'brickex'. [2011-05-20 10:02:41.821814] I [client-handshake.c:1014:select_server_supported_programs] 1-172.26.98.59-1: Using Program GlusterFS-3.1.0, Num (1298437), Version (310) [2011-05-20 10:02:41.821852] I [client-handshake.c:852:client_setvolume_cbk] 1-172.26.98.57-1: Connected to 172.26.98.57:6996, attached to remote volume 'brickex'. [2011-05-20 10:02:41.821863] I [afr-common.c:2552:afr_notify] 1-replicate-2: Subvolume '172.26.98.57-1' came back up; going online. [2011-05-20 10:02:41.824742] I [fuse-bridge.c:3316:fuse_graph_setup] 0-fuse: switched graph to 1 [2011-05-20 10:02:41.824784] I [client-handshake.c:852:client_setvolume_cbk] 1-172.26.98.59-1: Connected to 172.26.98.59:6996, attached to remote volume 'brickex'. [2011-05-20 10:02:42.832141] I [afr-common.c:819:afr_fresh_lookup_cbk] 1-replicate-1: added root inode [2011-05-20 10:02:42.832656] I [afr-common.c:819:afr_fresh_lookup_cbk] 1-replicate-2: added root inode [2011-05-20 10:02:42.847511] I [afr-common.c:716:afr_lookup_done] 1-replicate-2: background data self-heal triggered. path: /logs/admin_scheduler_lock [2011-05-20 10:02:42.849009] I [afr-self-heal-common.c:1527:afr_self_heal_completion_cbk] 1-replicate-2: background data self-heal completed on /logs/admin_scheduler_lock [2011-05-20 10:06:41.459183] W [fuse-bridge.c:582:fuse_fd_cbk] 0-glusterfs-fuse: 333260: OPEN() /testdir/input11.idx => -1 (Invalid argument) [2011-05-20 10:06:41.462296] W [fuse-bridge.c:582:fuse_fd_cbk] 0-glusterfs-fuse: 333267: OPEN() /testdir/input13.idx => -1 (Invalid argument) [2011-05-20 10:06:41.464954] W [fuse-bridge.c:582:fuse_fd_cbk] 0-glusterfs-fuse: 333274: OPEN() /testdir/input15.idx => -1 (Invalid argument) [2011-05-20 10:06:41.465279] W [fuse-bridge.c:582:fuse_fd_cbk] 0-glusterfs-fuse: 333275: OPEN() /testdir/input16.idx => -1 (Invalid argument) [2011-05-20 10:06:41.465586] W [fuse-bridge.c:582:fuse_fd_cbk] 0-glusterfs-fuse: 333276: OPEN() /testdir/input17.idx => -1 (Invalid argument) [2011-05-20 10:06:41.465915] W [fuse-bridge.c:582:fuse_fd_cbk] 0-glusterfs-fuse: 333277: OPEN() /testdir/input18.idx => -1 (Invalid argument) [2011-05-20 10:06:41.480038] W [fuse-bridge.c:582:fuse_fd_cbk] 0-glusterfs-fuse: 333319: OPEN() /testdir/input11.idx => -1 (Invalid argument) tx Vikas -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://gluster.org/pipermail/gluster-users/attachments/20110520/604a2c82/attachment-0001.htm> -------------- next part -------------- A non-text attachment was scrubbed... Name: glusterfsc.vol Type: application/octet-stream Size: 2504 bytes Desc: not available URL: <http://gluster.org/pipermail/gluster-users/attachments/20110520/604a2c82/attachment-0003.obj> -------------- next part -------------- A non-text attachment was scrubbed... Name: glusterfsd.vol Type: application/octet-stream Size: 827 bytes Desc: not available URL: <http://gluster.org/pipermail/gluster-users/attachments/20110520/604a2c82/attachment-0004.obj> -------------- next part -------------- A non-text attachment was scrubbed... Name: repro.py Type: application/octet-stream Size: 3413 bytes Desc: not available URL: <http://gluster.org/pipermail/gluster-users/attachments/20110520/604a2c82/attachment-0005.obj>