Soumya, We just added the nolock NFS option on the client and it's now working with sybase backup process. I wonder what could affect the client working on gluster NFS and if the nolock option does any bad for gluster. Thanks Peter ________________________________________ From: gluster-users-bounces@xxxxxxxxxxx [gluster-users-bounces@xxxxxxxxxxx] on behalf of Peter Auyeung [pauyeung@xxxxxxxxxxxxx] Sent: Friday, January 23, 2015 10:05 AM To: Soumya Koduri; gluster-devel@xxxxxxxxxxx; gluster-users@xxxxxxxxxxx Subject: Re: [Gluster-devel] [Gluster-user] Sybase backup server failed to write to Gluster NFS Hi Soumya, Yes this is strange as the same group of sybase servers been able to write their backup to gluster for the last 3 months. They can still write as sybase user on OS but not the sybase backup server process. They were able to mount another NFS server and perform both sybase OS user read/write and sybase backup server process. Here are the NFS debug log while we tried to perform backup via sybase process to gluster Jan 21 14:08:46 repdb006 kernel: NFS reply getattr Jan 21 14:08:46 repdb006 kernel: NFS call setacl Jan 21 14:08:46 repdb006 kernel: NFS reply setacl: 0 Jan 21 14:08:46 repdb006 kernel: NFS call lookup test~ Jan 21 14:08:46 repdb006 kernel: NFS reply lookup: 0 Jan 21 14:08:46 repdb006 kernel: NFS call remove test~ Jan 21 14:08:46 repdb006 kernel: NFS reply remove: 0 Jan 21 14:08:46 repdb006 kernel: NFS: nfs3_forget_cached_acls(0:13/-1909463086) Jan 21 14:08:46 repdb006 kernel: NFS call getattr Jan 21 14:08:46 repdb006 kernel: NFS reply getattr Jan 21 14:08:46 repdb006 kernel: NFS: nfs3_forget_cached_acls(0:13/1099635544) Jan 21 14:08:47 repdb006 kernel: NFS call getattr Jan 21 14:08:47 repdb006 kernel: NFS reply getattr Jan 21 14:08:47 repdb006 kernel: NFS: nfs3_forget_cached_acls(0:13/-1009531860) Jan 21 14:08:47 repdb006 kernel: NFS call access Jan 21 14:08:47 repdb006 kernel: NFS reply access, status = 0 Jan 21 14:08:47 repdb006 kernel: NFS call lookup .test.swp Jan 21 14:08:47 repdb006 kernel: NFS reply lookup: 0 Jan 21 14:08:47 repdb006 kernel: NFS call remove .test.swp Jan 21 14:08:47 repdb006 kernel: NFS reply remove: 0 Jan 21 14:08:47 repdb006 kernel: NFS: nfs3_forget_cached_acls(0:13/1099635544) Jan 21 14:08:48 repdb006 kernel: NFS call access Jan 21 14:08:48 repdb006 kernel: NFS reply access, status = 0 Jan 21 14:08:48 repdb006 kernel: NFS call getattr Jan 21 14:08:48 repdb006 kernel: NFS reply getattr Jan 21 14:08:48 repdb006 kernel: NFS: nfs3_forget_cached_acls(0:13/-1009531860) Jan 21 14:08:48 repdb006 kernel: NFS call access Jan 21 14:08:48 repdb006 kernel: NFS reply access, status = 0 Jan 21 14:08:48 repdb006 kernel: NFS call readdirplus 0 Jan 21 14:08:48 repdb006 kernel: NFS reply readdir: 2 Jan 21 14:08:48 repdb006 kernel: NFS call readdirplus 37 Jan 21 14:08:48 repdb006 kernel: NFS reply readdir: 1 Jan 21 14:08:48 repdb006 kernel: NFS call readdirplus 3073 Jan 21 14:08:48 repdb006 kernel: NFS reply readdir: 1 Jan 21 14:08:48 repdb006 kernel: NFS call readdirplus 3074 Jan 21 14:08:48 repdb006 kernel: NFS reply readdir: 1 Jan 21 14:08:48 repdb006 kernel: NFS call readdirplus 3075 Jan 21 14:08:48 repdb006 kernel: NFS reply readdir: 1 Jan 21 14:08:48 repdb006 kernel: NFS call readdirplus 3076 Jan 21 14:08:48 repdb006 kernel: NFS reply readdir: 0 Jan 21 14:08:48 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/-1009531860, 32768) = fffffff5 Jan 21 14:08:48 repdb006 kernel: NFS call getacl Jan 21 14:08:48 repdb006 kernel: NFS reply getacl: 0 Jan 21 14:08:48 repdb006 kernel: NFS call getattr Jan 21 14:08:48 repdb006 kernel: NFS reply getattr Jan 21 14:08:48 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/-1009531860, 16384) = 00000000 Jan 21 14:08:48 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/487151148, 32768) = 00000000 Jan 21 14:08:48 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/487151148, 16384) = 00000000 Jan 21 14:08:48 repdb006 kernel: NFS call getattr Jan 21 14:08:48 repdb006 kernel: NFS reply getattr Jan 21 14:08:48 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/-1991882046, 32768) = 00000000 Jan 21 14:08:48 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/-1991882046, 16384) = 00000000 Jan 21 14:08:48 repdb006 kernel: NFS call getattr Jan 21 14:08:48 repdb006 kernel: NFS reply getattr Jan 21 14:08:48 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/1247752239, 32768) = 00000000 Jan 21 14:08:48 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/1247752239, 16384) = 00000000 Jan 21 14:08:48 repdb006 kernel: NFS call getattr Jan 21 14:08:48 repdb006 kernel: NFS reply getattr Jan 21 14:08:48 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/2137268371, 32768) = 00000000 Jan 21 14:08:48 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/2137268371, 16384) = 00000000 Jan 21 14:08:48 repdb006 kernel: NFS call getattr Jan 21 14:08:48 repdb006 kernel: NFS reply getattr Jan 21 14:08:48 repdb006 kernel: NFS: nfs3_forget_cached_acls(0:13/-702684555) Jan 21 14:08:48 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/-702684555, 32768) = fffffff5 Jan 21 14:08:48 repdb006 kernel: NFS call getacl Jan 21 14:08:48 repdb006 kernel: NFS reply getacl: 0 Jan 21 14:08:49 repdb006 kernel: NFS call getattr Jan 21 14:08:49 repdb006 kernel: NFS reply getattr Jan 21 14:08:49 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/-702684555, 16384) = 00000000 Jan 21 14:09:31 repdb006 kernel: NFS call getattr Jan 21 14:09:31 repdb006 kernel: NFS reply getattr Jan 21 14:09:31 repdb006 kernel: NFS: nfs3_forget_cached_acls(0:13/-1964955375) Jan 21 14:09:31 repdb006 kernel: NFS call access Jan 21 14:09:31 repdb006 kernel: NFS reply access, status = 0 Jan 21 14:09:31 repdb006 kernel: NFS call getattr Jan 21 14:09:31 repdb006 kernel: NFS reply getattr Jan 21 14:09:31 repdb006 kernel: NFS: nfs3_forget_cached_acls(0:13/487151148) Jan 21 14:09:31 repdb006 kernel: NFS call access Jan 21 14:09:31 repdb006 kernel: NFS reply access, status = 0 Jan 21 14:09:31 repdb006 kernel: NFS call getattr Jan 21 14:09:31 repdb006 kernel: NFS reply getattr Jan 21 14:09:33 repdb006 kernel: NFS call getattr Jan 21 14:09:33 repdb006 kernel: NFS reply getattr Jan 21 14:09:33 repdb006 kernel: NFS call getattr Jan 21 14:09:33 repdb006 kernel: NFS reply getattr Jan 21 14:09:33 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/-1991882046, 32768) = 00000000 Jan 21 14:09:33 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/-1991882046, 16384) = 00000000 Jan 21 14:09:33 repdb006 kernel: NFS call getattr Jan 21 14:09:33 repdb006 kernel: NFS reply getattr Jan 21 14:09:33 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/1247752239, 32768) = 00000000 Jan 21 14:09:33 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/1247752239, 16384) = 00000000 Jan 21 14:09:33 repdb006 kernel: NFS call getattr Jan 21 14:09:33 repdb006 kernel: NFS reply getattr Jan 21 14:09:33 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/2137268371, 32768) = 00000000 Jan 21 14:09:33 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/2137268371, 16384) = 00000000 Jan 21 14:09:33 repdb006 kernel: NFS call getattr Jan 21 14:09:33 repdb006 kernel: NFS reply getattr Jan 21 14:09:33 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/-702684555, 32768) = 00000000 Jan 21 14:09:33 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/-702684555, 16384) = 00000000 Jan 21 14:09:40 repdb006 kernel: NFS call getattr Jan 21 14:09:40 repdb006 kernel: NFS reply getattr Jan 21 14:09:40 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/-1009531860, 32768) = 00000000 Jan 21 14:09:40 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/-1009531860, 16384) = 00000000 Jan 21 14:09:40 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/487151148, 32768) = fffffff5 Jan 21 14:09:40 repdb006 kernel: NFS call getacl Jan 21 14:09:40 repdb006 kernel: NFS reply getacl: 0 Jan 21 14:09:40 repdb006 kernel: NFS call getattr Jan 21 14:09:40 repdb006 kernel: NFS reply getattr Jan 21 14:09:40 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/487151148, 16384) = 00000000 Jan 21 14:09:40 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/-1991882046, 32768) = 00000000 Jan 21 14:09:40 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/-1991882046, 16384) = 00000000 Jan 21 14:09:40 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/1247752239, 32768) = 00000000 Jan 21 14:09:40 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/1247752239, 16384) = 00000000 Jan 21 14:09:40 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/2137268371, 32768) = 00000000 Jan 21 14:09:40 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/2137268371, 16384) = 00000000 Jan 21 14:09:40 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/-702684555, 32768) = 00000000 Jan 21 14:09:40 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/-702684555, 16384) = 00000000 Jan 21 14:10:16 repdb006 kernel: NFS call access Jan 21 14:10:16 repdb006 kernel: NFS reply access, status = 0 Jan 21 14:10:16 repdb006 kernel: NFS call lookup pr_rssd_dss_master.jz.111 Jan 21 14:10:16 repdb006 kernel: NFS reply lookup: -2 Jan 21 14:10:16 repdb006 kernel: NFS call create pr_rssd_dss_master.jz.111 Jan 21 14:10:16 repdb006 kernel: NFS call getattr Jan 21 14:10:16 repdb006 kernel: NFS reply getattr Jan 21 14:10:16 repdb006 kernel: NFS: nfs3_forget_cached_acls(0:13/-1009531860) Jan 21 14:10:16 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/-1009531860, 16384) = fffffff5 Jan 21 14:10:16 repdb006 kernel: NFS call getacl Jan 21 14:10:16 repdb006 kernel: NFS reply getacl: 0 Jan 21 14:10:16 repdb006 kernel: NFS reply create: 0 Jan 21 14:10:16 repdb006 kernel: NFS call getattr Jan 21 14:10:16 repdb006 kernel: NFS reply getattr Jan 21 14:10:16 repdb006 kernel: NFS call access Jan 21 14:10:16 repdb006 kernel: NFS reply access, status = 0 Jan 21 14:10:16 repdb006 kernel: NFS call lookup pr_rssd_dss_master.jz.111 Jan 21 14:10:16 repdb006 kernel: NFS reply lookup: 0 Jan 21 14:10:16 repdb006 kernel: NFS call access Jan 21 14:10:16 repdb006 kernel: NFS reply access, status = 0 Jan 21 14:10:16 repdb006 kernel: NFS call setattr Jan 21 14:10:17 repdb006 kernel: NFS reply setattr Jan 21 14:10:23 repdb006 kernel: NFS call access Jan 21 14:10:23 repdb006 kernel: NFS reply access, status = 0 Jan 21 14:10:23 repdb006 kernel: NFS call getattr Jan 21 14:10:23 repdb006 kernel: NFS reply getattr Jan 21 14:10:23 repdb006 kernel: NFS: nfs3_forget_cached_acls(0:13/-1009531860) Jan 21 14:10:23 repdb006 kernel: NFS call access Jan 21 14:10:23 repdb006 kernel: NFS reply access, status = 0 Jan 21 14:10:23 repdb006 kernel: NFS call readdirplus 0 Jan 21 14:10:23 repdb006 kernel: NFS reply readdir: 2 Jan 21 14:10:23 repdb006 kernel: NFS call readdirplus 37 Jan 21 14:10:23 repdb006 kernel: NFS reply readdir: 1 Jan 21 14:10:23 repdb006 kernel: NFS call readdirplus 3073 Jan 21 14:10:23 repdb006 kernel: NFS reply readdir: 1 Jan 21 14:10:23 repdb006 kernel: NFS call readdirplus 3074 Jan 21 14:10:23 repdb006 kernel: NFS reply readdir: 2 Jan 21 14:10:23 repdb006 kernel: NFS call readdirplus 3075 Jan 21 14:10:23 repdb006 kernel: NFS reply readdir: 1 Jan 21 14:10:23 repdb006 kernel: NFS call readdirplus 3076 Jan 21 14:10:23 repdb006 kernel: NFS reply readdir: 0 Jan 21 14:10:23 repdb006 kernel: NFS call getattr Jan 21 14:10:23 repdb006 kernel: NFS reply getattr Jan 21 14:10:23 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/-1991882046, 32768) = 00000000 Jan 21 14:10:23 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/-1991882046, 16384) = 00000000 Jan 21 14:10:23 repdb006 kernel: NFS call getattr Jan 21 14:10:23 repdb006 kernel: NFS reply getattr Jan 21 14:10:23 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/1247752239, 32768) = 00000000 Jan 21 14:10:23 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/1247752239, 16384) = 00000000 Jan 21 14:10:23 repdb006 kernel: NFS call getattr Jan 21 14:10:23 repdb006 kernel: NFS reply getattr Jan 21 14:10:23 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/2137268371, 32768) = 00000000 Jan 21 14:10:23 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/2137268371, 16384) = 00000000 Jan 21 14:10:23 repdb006 kernel: NFS call getattr Jan 21 14:10:23 repdb006 kernel: NFS reply getattr Jan 21 14:10:23 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/1544741310, 32768) = fffffff5 Jan 21 14:10:23 repdb006 kernel: NFS call getacl Jan 21 14:10:23 repdb006 kernel: NFS reply getacl: 0 Jan 21 14:10:23 repdb006 kernel: NFS call getattr Jan 21 14:10:23 repdb006 kernel: NFS reply getattr Jan 21 14:10:23 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/1544741310, 16384) = 00000000 Jan 21 14:10:23 repdb006 kernel: NFS call getattr Jan 21 14:10:23 repdb006 kernel: NFS reply getattr Jan 21 14:10:23 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/-702684555, 32768) = 00000000 Jan 21 14:10:23 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/-702684555, 16384) = 00000000 Jan 21 14:11:22 repdb006 kernel: NFS call getattr Jan 21 14:11:22 repdb006 kernel: NFS reply getattr Jan 21 14:12:38 repdb006 kernel: NFS call access Jan 21 14:12:39 repdb006 kernel: NFS reply access, status = 0 Jan 21 14:12:39 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:12:39 repdb006 kernel: NFS: revalidating (0:13/-8058565423364083875) Jan 21 14:12:39 repdb006 kernel: NFS call getattr Jan 21 14:12:40 repdb006 kernel: NFS reply getattr Jan 21 14:12:40 repdb006 kernel: NFS: nfs_update_inode(0:13/487151148 ct=1 info=0x6) Jan 21 14:12:40 repdb006 kernel: NFS: (0:13/-8058565423364083875) revalidation complete Jan 21 14:12:40 repdb006 kernel: NFS: revalidating (0:13/-6148448765687654848) Jan 21 14:12:40 repdb006 kernel: NFS call getattr Jan 21 14:12:41 repdb006 kernel: NFS reply getattr Jan 21 14:12:41 repdb006 kernel: NFS: nfs_update_inode(0:13/-1009531860 ct=1 info=0x6) Jan 21 14:12:41 repdb006 kernel: NFS: (0:13/-6148448765687654848) revalidation complete Jan 21 14:12:41 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x4, res=0 Jan 21 14:12:41 repdb006 kernel: NFS: readdir_search_pagecache() searching for offset 0 Jan 21 14:12:41 repdb006 kernel: NFS: find_dirent_page() searching directory page 0 Jan 21 14:12:41 repdb006 kernel: NFS: found cookie 25 at index 0 Jan 21 14:12:41 repdb006 kernel: NFS: find_dirent_index() returns 0 Jan 21 14:12:41 repdb006 kernel: NFS: find_dirent_page() returns 0 Jan 21 14:12:41 repdb006 kernel: NFS: readdir_search_pagecache() returned 0 Jan 21 14:12:41 repdb006 kernel: NFS: nfs_do_filldir() filling starting @ cookie 25 Jan 21 14:12:41 repdb006 kernel: NFS: nfs_do_filldir() filling ended @ cookie 37; returning = 0 Jan 21 14:12:41 repdb006 kernel: NFS: readdir_search_pagecache() searching for cookie 37 Jan 21 14:12:41 repdb006 kernel: NFS: find_dirent_page() searching directory page 1 Jan 21 14:12:41 repdb006 kernel: NFS: found cookie 3073 Jan 21 14:12:41 repdb006 kernel: NFS: find_dirent() returns 0 Jan 21 14:12:41 repdb006 kernel: NFS: find_dirent_page() returns 0 Jan 21 14:12:41 repdb006 kernel: NFS: readdir_search_pagecache() returned 0 Jan 21 14:12:41 repdb006 kernel: NFS: nfs_do_filldir() filling starting @ cookie 3073 Jan 21 14:12:41 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz.11, 8) Jan 21 14:12:41 repdb006 kernel: NFS: nfs_do_filldir() filling ended @ cookie 3073; returning = 0 Jan 21 14:12:41 repdb006 kernel: NFS: readdir_search_pagecache() searching for cookie 3073 Jan 21 14:12:41 repdb006 kernel: NFS: find_dirent_page() searching directory page 2 Jan 21 14:12:41 repdb006 kernel: NFS: found cookie 3074 Jan 21 14:12:41 repdb006 kernel: NFS: find_dirent() returns 0 Jan 21 14:12:41 repdb006 kernel: NFS: find_dirent_page() returns 0 Jan 21 14:12:41 repdb006 kernel: NFS: readdir_search_pagecache() returned 0 Jan 21 14:12:41 repdb006 kernel: NFS: nfs_do_filldir() filling starting @ cookie 3074 Jan 21 14:12:41 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz.1, 8) Jan 21 14:12:41 repdb006 kernel: NFS: nfs_do_filldir() filling ended @ cookie 3074; returning = 0 Jan 21 14:12:41 repdb006 kernel: NFS: readdir_search_pagecache() searching for cookie 3074 Jan 21 14:12:41 repdb006 kernel: NFS: find_dirent_page() searching directory page 3 Jan 21 14:12:41 repdb006 kernel: NFS: found cookie 63 Jan 21 14:12:41 repdb006 kernel: NFS: find_dirent() returns 0 Jan 21 14:12:41 repdb006 kernel: NFS: find_dirent_page() returns 0 Jan 21 14:12:41 repdb006 kernel: NFS: readdir_search_pagecache() returned 0 Jan 21 14:12:41 repdb006 kernel: NFS: nfs_do_filldir() filling starting @ cookie 63 Jan 21 14:12:41 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz, 8) Jan 21 14:12:41 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz.111, 8) Jan 21 14:12:41 repdb006 kernel: NFS: nfs_do_filldir() filling ended @ cookie 3075; returning = 0 Jan 21 14:12:41 repdb006 kernel: NFS: readdir_search_pagecache() searching for cookie 3075 Jan 21 14:12:41 repdb006 kernel: NFS: find_dirent_page() searching directory page 4 Jan 21 14:12:41 repdb006 kernel: NFS: found cookie 3076 Jan 21 14:12:41 repdb006 kernel: NFS: find_dirent() returns 0 Jan 21 14:12:41 repdb006 kernel: NFS: find_dirent_page() returns 0 Jan 21 14:12:41 repdb006 kernel: NFS: readdir_search_pagecache() returned 0 Jan 21 14:12:41 repdb006 kernel: NFS: nfs_do_filldir() filling starting @ cookie 3076 Jan 21 14:12:41 repdb006 kernel: NFS: dentry_delete(full/test, 8) Jan 21 14:12:41 repdb006 kernel: NFS: nfs_do_filldir() filling ended @ cookie 3076; returning = 0 Jan 21 14:12:42 repdb006 kernel: NFS: readdir_search_pagecache() searching for cookie 3076 Jan 21 14:12:42 repdb006 kernel: NFS: find_dirent_page() searching directory page 5 Jan 21 14:12:42 repdb006 kernel: NFS: find_dirent() returns -523 Jan 21 14:12:42 repdb006 kernel: NFS: find_dirent_page() returns -523 Jan 21 14:12:42 repdb006 kernel: NFS: readdir_search_pagecache() returned -523 Jan 21 14:12:42 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:12:42 repdb006 kernel: NFS: revalidating (0:13/-5879706972258738325) Jan 21 14:12:42 repdb006 kernel: NFS call getattr Jan 21 14:12:42 repdb006 kernel: NFS reply getattr Jan 21 14:12:42 repdb006 kernel: NFS: nfs_update_inode(0:13/-1991882046 ct=1 info=0x6) Jan 21 14:12:42 repdb006 kernel: NFS: (0:13/-5879706972258738325) revalidation complete Jan 21 14:12:42 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz.11, 8) Jan 21 14:12:42 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:12:42 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/-1991882046, 32768) = 00000000 Jan 21 14:12:42 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz.11, 8) Jan 21 14:12:42 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:12:43 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/-1991882046, 16384) = 00000000 Jan 21 14:12:43 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz.11, 8) Jan 21 14:12:43 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:12:43 repdb006 kernel: NFS: revalidating (0:13/-8660661468576868760) Jan 21 14:12:43 repdb006 kernel: NFS call getattr Jan 21 14:12:43 repdb006 kernel: NFS reply getattr Jan 21 14:12:43 repdb006 kernel: NFS: nfs_update_inode(0:13/1247752239 ct=1 info=0x6) Jan 21 14:12:43 repdb006 kernel: NFS: (0:13/-8660661468576868760) revalidation complete Jan 21 14:12:43 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz.1, 8) Jan 21 14:12:44 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:12:44 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/1247752239, 32768) = 00000000 Jan 21 14:12:44 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz.1, 8) Jan 21 14:12:44 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:12:44 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/1247752239, 16384) = 00000000 Jan 21 14:12:44 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz.1, 8) Jan 21 14:12:44 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:12:44 repdb006 kernel: NFS: revalidating (0:13/-8417256840809953812) Jan 21 14:12:44 repdb006 kernel: NFS call getattr Jan 21 14:12:44 repdb006 kernel: NFS reply getattr Jan 21 14:12:44 repdb006 kernel: NFS: nfs_update_inode(0:13/2137268371 ct=1 info=0x6) Jan 21 14:12:44 repdb006 kernel: NFS: (0:13/-8417256840809953812) revalidation complete Jan 21 14:12:44 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz, 8) Jan 21 14:12:44 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:12:44 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/2137268371, 32768) = 00000000 Jan 21 14:12:44 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz, 8) Jan 21 14:12:45 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:12:45 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/2137268371, 16384) = 00000000 Jan 21 14:12:45 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz, 8) Jan 21 14:12:45 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:12:45 repdb006 kernel: NFS: revalidating (0:13/-5992023206934155491) Jan 21 14:12:45 repdb006 kernel: NFS call getattr Jan 21 14:12:45 repdb006 kernel: NFS reply getattr Jan 21 14:12:45 repdb006 kernel: NFS: nfs_update_inode(0:13/1544741310 ct=1 info=0x6) Jan 21 14:12:45 repdb006 kernel: NFS: (0:13/-5992023206934155491) revalidation complete Jan 21 14:12:45 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz.111, 8) Jan 21 14:12:46 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:12:46 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/1544741310, 32768) = 00000000 Jan 21 14:12:46 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz.111, 8) Jan 21 14:12:46 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:12:46 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/1544741310, 16384) = 00000000 Jan 21 14:12:46 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz.111, 8) Jan 21 14:12:46 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:12:46 repdb006 kernel: NFS: revalidating (0:13/-5887948837043818545) Jan 21 14:12:46 repdb006 kernel: NFS call getattr Jan 21 14:12:46 repdb006 kernel: NFS reply getattr Jan 21 14:12:46 repdb006 kernel: NFS: nfs_update_inode(0:13/-702684555 ct=1 info=0x6) Jan 21 14:12:47 repdb006 kernel: NFS: (0:13/-5887948837043818545) revalidation complete Jan 21 14:12:47 repdb006 kernel: NFS: dentry_delete(full/test, 8) Jan 21 14:12:47 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:12:47 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/-702684555, 32768) = 00000000 Jan 21 14:12:47 repdb006 kernel: NFS: dentry_delete(full/test, 8) Jan 21 14:12:47 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:12:47 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/-702684555, 16384) = 00000000 Jan 21 14:12:47 repdb006 kernel: NFS: dentry_delete(full/test, 8) Jan 21 14:12:47 repdb006 kernel: NFS: readdir_search_pagecache() searching for cookie 3076 Jan 21 14:12:47 repdb006 kernel: NFS: find_dirent_page() searching directory page 0 Jan 21 14:12:47 repdb006 kernel: NFS: found cookie 25 Jan 21 14:12:47 repdb006 kernel: NFS: found cookie 37 Jan 21 14:12:47 repdb006 kernel: NFS: find_dirent() returns -11 Jan 21 14:12:47 repdb006 kernel: NFS: find_dirent_page() returns -11 Jan 21 14:12:47 repdb006 kernel: NFS: find_dirent_page() searching directory page 1 Jan 21 14:12:47 repdb006 kernel: NFS: found cookie 3073 Jan 21 14:12:47 repdb006 kernel: NFS: find_dirent() returns -11 Jan 21 14:12:47 repdb006 kernel: NFS: find_dirent_page() returns -11 Jan 21 14:12:47 repdb006 kernel: NFS: find_dirent_page() searching directory page 2 Jan 21 14:12:47 repdb006 kernel: NFS: found cookie 3074 Jan 21 14:12:47 repdb006 kernel: NFS: find_dirent() returns -11 Jan 21 14:12:47 repdb006 kernel: NFS: find_dirent_page() returns -11 Jan 21 14:12:47 repdb006 kernel: NFS: find_dirent_page() searching directory page 3 Jan 21 14:12:47 repdb006 kernel: NFS: found cookie 63 Jan 21 14:12:47 repdb006 kernel: NFS: found cookie 3075 Jan 21 14:12:47 repdb006 kernel: NFS: find_dirent() returns -11 Jan 21 14:12:47 repdb006 kernel: NFS: find_dirent_page() returns -11 Jan 21 14:12:47 repdb006 kernel: NFS: find_dirent_page() searching directory page 4 Jan 21 14:12:47 repdb006 kernel: NFS: found cookie 3076 Jan 21 14:12:47 repdb006 kernel: NFS: find_dirent() returns -11 Jan 21 14:12:47 repdb006 kernel: NFS: find_dirent_page() returns -11 Jan 21 14:12:47 repdb006 kernel: NFS: find_dirent_page() searching directory page 5 Jan 21 14:12:47 repdb006 kernel: NFS: find_dirent() returns -523 Jan 21 14:12:47 repdb006 kernel: NFS: find_dirent_page() returns -523 Jan 21 14:12:47 repdb006 kernel: NFS: readdir_search_pagecache() returned -523 Jan 21 14:12:57 repdb006 kernel: NFS call access Jan 21 14:12:58 repdb006 kernel: NFS reply access, status = 0 Jan 21 14:12:58 repdb006 kernel: NFS: permission(0:13/-1964955375), mask=0x1, res=0 Jan 21 14:12:58 repdb006 kernel: NFS: revalidating (0:13/-6006060748298058109) Jan 21 14:12:58 repdb006 kernel: NFS call getattr Jan 21 14:12:59 repdb006 kernel: NFS reply getattr Jan 21 14:12:59 repdb006 kernel: NFS: nfs_update_inode(0:13/-1964955375 ct=1 info=0x6) Jan 21 14:12:59 repdb006 kernel: NFS: (0:13/-6006060748298058109) revalidation complete Jan 21 14:12:59 repdb006 kernel: NFS call access Jan 21 14:13:00 repdb006 kernel: NFS reply access, status = 0 Jan 21 14:13:00 repdb006 kernel: NFS: permission(0:13/487151148), mask=0x1, res=0 Jan 21 14:13:00 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:13:00 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz, 8) Jan 21 14:13:55 repdb006 kernel: NFS: permission(0:13/-1964955375), mask=0x1, res=0 Jan 21 14:13:55 repdb006 kernel: NFS: revalidating (0:13/-8058565423364083875) Jan 21 14:13:55 repdb006 kernel: NFS call getattr Jan 21 14:13:55 repdb006 kernel: NFS reply getattr Jan 21 14:13:55 repdb006 kernel: NFS: nfs_update_inode(0:13/487151148 ct=1 info=0x6) Jan 21 14:13:55 repdb006 kernel: NFS: (0:13/-8058565423364083875) revalidation complete Jan 21 14:13:55 repdb006 kernel: NFS: permission(0:13/487151148), mask=0x1, res=0 Jan 21 14:13:55 repdb006 kernel: NFS: revalidating (0:13/-6148448765687654848) Jan 21 14:13:55 repdb006 kernel: NFS call getattr Jan 21 14:13:56 repdb006 kernel: NFS reply getattr Jan 21 14:13:56 repdb006 kernel: NFS: nfs_update_inode(0:13/-1009531860 ct=1 info=0x6) Jan 21 14:13:56 repdb006 kernel: NFS: (0:13/-6148448765687654848) revalidation complete Jan 21 14:13:56 repdb006 kernel: NFS call access Jan 21 14:13:57 repdb006 kernel: NFS reply access, status = 0 Jan 21 14:13:57 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:13:57 repdb006 kernel: NFS: lookup(full/pr_rssd_dss_master.jz.2) Jan 21 14:13:57 repdb006 kernel: NFS call lookup pr_rssd_dss_master.jz.2 Jan 21 14:13:58 repdb006 kernel: NFS reply lookup: -2 Jan 21 14:13:58 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz.2, 0) Jan 21 14:13:58 repdb006 kernel: NFS: permission(0:13/-1964955375), mask=0x1, res=0 Jan 21 14:13:58 repdb006 kernel: NFS: permission(0:13/487151148), mask=0x1, res=0 Jan 21 14:13:58 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:13:58 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:13:58 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x3, res=0 Jan 21 14:13:58 repdb006 kernel: NFS: create(0:13/-1009531860, pr_rssd_dss_master.jz.2 Jan 21 14:13:58 repdb006 kernel: NFS call create pr_rssd_dss_master.jz.2 Jan 21 14:13:59 repdb006 kernel: NFS: nfs_update_inode(0:13/-1009531860 ct=1 info=0x7) Jan 21 14:13:59 repdb006 kernel: NFS: isize change on server for file 0:13/-1009531860 Jan 21 14:13:59 repdb006 kernel: NFS: mtime change on server for file 0:13/-1009531860 Jan 21 14:13:59 repdb006 kernel: NFS: nfs_fhget(0:13/-8936416646988302039 ct=1) Jan 21 14:13:59 repdb006 kernel: NFS: revalidating (0:13/-6148448765687654848) Jan 21 14:13:59 repdb006 kernel: NFS call getattr Jan 21 14:13:59 repdb006 kernel: NFS reply getattr Jan 21 14:14:00 repdb006 kernel: NFS: nfs_update_inode(0:13/-1009531860 ct=1 info=0x6) Jan 21 14:14:00 repdb006 kernel: NFS: isize change on server for file 0:13/-1009531860 Jan 21 14:14:00 repdb006 kernel: NFS: mtime change on server for file 0:13/-1009531860 Jan 21 14:14:00 repdb006 kernel: NFS: (0:13/-6148448765687654848) data cache invalidated Jan 21 14:14:00 repdb006 kernel: NFS: nfs3_forget_cached_acls(0:13/-1009531860) Jan 21 14:14:00 repdb006 kernel: NFS: (0:13/-6148448765687654848) revalidation complete Jan 21 14:14:00 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/-1009531860, 16384) = fffffff5 Jan 21 14:14:00 repdb006 kernel: NFS call getacl Jan 21 14:14:00 repdb006 kernel: NFS reply getacl: 0 Jan 21 14:14:00 repdb006 kernel: NFS reply create: 0 Jan 21 14:14:00 repdb006 kernel: NFS: permission(0:13/-1666940607), mask=0x0, res=0 Jan 21 14:14:00 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz.2, 8) Jan 21 14:14:00 repdb006 kernel: NFS: permission(0:13/-1964955375), mask=0x1, res=0 Jan 21 14:14:01 repdb006 kernel: NFS: permission(0:13/487151148), mask=0x1, res=0 Jan 21 14:14:01 repdb006 kernel: NFS: revalidating (0:13/-6148448765687654848) Jan 21 14:14:01 repdb006 kernel: NFS call getattr Jan 21 14:14:01 repdb006 kernel: NFS reply getattr Jan 21 14:14:01 repdb006 kernel: NFS: nfs_update_inode(0:13/-1009531860 ct=1 info=0x6) Jan 21 14:14:02 repdb006 kernel: NFS: isize change on server for file 0:13/-1009531860 Jan 21 14:14:02 repdb006 kernel: NFS: (0:13/-6148448765687654848) data cache invalidated Jan 21 14:14:02 repdb006 kernel: NFS: (0:13/-6148448765687654848) revalidation complete Jan 21 14:14:02 repdb006 kernel: NFS call access Jan 21 14:14:02 repdb006 kernel: NFS reply access, status = 0 Jan 21 14:14:02 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:14:02 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:14:02 repdb006 kernel: NFS call lookup pr_rssd_dss_master.jz.2 Jan 21 14:14:03 repdb006 kernel: NFS reply lookup: 0 Jan 21 14:14:03 repdb006 kernel: NFS: nfs_update_inode(0:13/-1009531860 ct=1 info=0x6) Jan 21 14:14:03 repdb006 kernel: NFS: isize change on server for file 0:13/-1009531860 Jan 21 14:14:03 repdb006 kernel: NFS: mtime change on server for file 0:13/-1009531860 Jan 21 14:14:03 repdb006 kernel: NFS: nfs_update_inode(0:13/-1666940607 ct=1 info=0x6) Jan 21 14:14:03 repdb006 kernel: NFS call access Jan 21 14:14:04 repdb006 kernel: NFS reply access, status = 0 Jan 21 14:14:04 repdb006 kernel: NFS: permission(0:13/-1666940607), mask=0x6, res=0 Jan 21 14:14:04 repdb006 kernel: NFS call setattr Jan 21 14:14:05 repdb006 kernel: NFS reply setattr Jan 21 14:14:05 repdb006 kernel: NFS: nfs_update_inode(0:13/-1666940607 ct=1 info=0x7) Jan 21 14:14:05 repdb006 kernel: NFS: nfs_lock(f=0:13/-1666940607, t=1, fl=1, r=0:9223372036854775807) Jan 21 14:14:12 repdb006 kernel: NFS call access Jan 21 14:14:12 repdb006 kernel: NFS reply access, status = 0 Jan 21 14:14:12 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:14:12 repdb006 kernel: NFS: revalidating (0:13/-6148448765687654848) Jan 21 14:14:12 repdb006 kernel: NFS call getattr Jan 21 14:14:13 repdb006 kernel: NFS reply getattr Jan 21 14:14:13 repdb006 kernel: NFS: nfs_update_inode(0:13/-1009531860 ct=1 info=0x6) Jan 21 14:14:13 repdb006 kernel: NFS: isize change on server for file 0:13/-1009531860 Jan 21 14:14:13 repdb006 kernel: NFS: mtime change on server for file 0:13/-1009531860 Jan 21 14:14:13 repdb006 kernel: NFS: (0:13/-6148448765687654848) data cache invalidated Jan 21 14:14:13 repdb006 kernel: NFS: nfs3_forget_cached_acls(0:13/-1009531860) Jan 21 14:14:13 repdb006 kernel: NFS: (0:13/-6148448765687654848) revalidation complete Jan 21 14:14:13 repdb006 kernel: NFS call access Jan 21 14:14:14 repdb006 kernel: NFS reply access, status = 0 Jan 21 14:14:14 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x4, res=0 Jan 21 14:14:14 repdb006 kernel: NFS: readdir_search_pagecache() searching for offset 0 Jan 21 14:14:14 repdb006 kernel: NFS: find_dirent_page() searching directory page 0 Jan 21 14:14:14 repdb006 kernel: NFS: nfs_readdir_filler() reading cookie 0 into page 0. Jan 21 14:14:14 repdb006 kernel: NFS call readdirplus 0 Jan 21 14:14:15 repdb006 kernel: NFS: nfs_update_inode(0:13/-1009531860 ct=1 info=0x6) Jan 21 14:14:15 repdb006 kernel: NFS reply readdir: 2 Jan 21 14:14:15 repdb006 kernel: NFS: found cookie 25 at index 0 Jan 21 14:14:15 repdb006 kernel: NFS: find_dirent_index() returns 0 Jan 21 14:14:15 repdb006 kernel: NFS: find_dirent_page() returns 0 Jan 21 14:14:15 repdb006 kernel: NFS: readdir_search_pagecache() returned 0 Jan 21 14:14:15 repdb006 kernel: NFS: nfs_do_filldir() filling starting @ cookie 25 Jan 21 14:14:15 repdb006 kernel: NFS: nfs_do_filldir() filling ended @ cookie 37; returning = 0 Jan 21 14:14:15 repdb006 kernel: NFS: readdir_search_pagecache() searching for cookie 37 Jan 21 14:14:15 repdb006 kernel: NFS: find_dirent_page() searching directory page 1 Jan 21 14:14:15 repdb006 kernel: NFS: nfs_readdir_filler() reading cookie 37 into page 1. Jan 21 14:14:15 repdb006 kernel: NFS call readdirplus 37 Jan 21 14:14:16 repdb006 kernel: NFS reply readdir: 1 Jan 21 14:14:16 repdb006 kernel: NFS: found cookie 3073 Jan 21 14:14:16 repdb006 kernel: NFS: find_dirent() returns 0 Jan 21 14:14:16 repdb006 kernel: NFS: find_dirent_page() returns 0 Jan 21 14:14:16 repdb006 kernel: NFS: readdir_search_pagecache() returned 0 Jan 21 14:14:16 repdb006 kernel: NFS: nfs_do_filldir() filling starting @ cookie 3073 Jan 21 14:14:16 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz.11, 8) Jan 21 14:14:16 repdb006 kernel: NFS: nfs_do_filldir() filling ended @ cookie 3073; returning = 0 Jan 21 14:14:16 repdb006 kernel: NFS: readdir_search_pagecache() searching for cookie 3073 Jan 21 14:14:16 repdb006 kernel: NFS: find_dirent_page() searching directory page 2 Jan 21 14:14:16 repdb006 kernel: NFS: nfs_readdir_filler() reading cookie 3073 into page 2. Jan 21 14:14:16 repdb006 kernel: NFS call readdirplus 3073 Jan 21 14:14:17 repdb006 kernel: NFS: nfs_update_inode(0:13/-1009531860 ct=1 info=0x6) Jan 21 14:14:17 repdb006 kernel: NFS reply readdir: 1 Jan 21 14:14:17 repdb006 kernel: NFS: found cookie 3074 Jan 21 14:14:17 repdb006 kernel: NFS: find_dirent() returns 0 Jan 21 14:14:17 repdb006 kernel: NFS: find_dirent_page() returns 0 Jan 21 14:14:17 repdb006 kernel: NFS: readdir_search_pagecache() returned 0 Jan 21 14:14:17 repdb006 kernel: NFS: nfs_do_filldir() filling starting @ cookie 3074 Jan 21 14:14:17 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz.1, 8) Jan 21 14:14:17 repdb006 kernel: NFS: nfs_do_filldir() filling ended @ cookie 3074; returning = 0 Jan 21 14:14:17 repdb006 kernel: NFS: readdir_search_pagecache() searching for cookie 3074 Jan 21 14:14:17 repdb006 kernel: NFS: find_dirent_page() searching directory page 3 Jan 21 14:14:17 repdb006 kernel: NFS: nfs_readdir_filler() reading cookie 3074 into page 3. Jan 21 14:14:17 repdb006 kernel: NFS call readdirplus 3074 Jan 21 14:14:17 repdb006 kernel: NFS: nfs_update_inode(0:13/-1009531860 ct=1 info=0x6) Jan 21 14:14:17 repdb006 kernel: NFS reply readdir: 2 Jan 21 14:14:17 repdb006 kernel: NFS: found cookie 63 Jan 21 14:14:17 repdb006 kernel: NFS: find_dirent() returns 0 Jan 21 14:14:17 repdb006 kernel: NFS: find_dirent_page() returns 0 Jan 21 14:14:17 repdb006 kernel: NFS: readdir_search_pagecache() returned 0 Jan 21 14:14:17 repdb006 kernel: NFS: nfs_do_filldir() filling starting @ cookie 63 Jan 21 14:14:17 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz, 8) Jan 21 14:14:18 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz.111, 8) Jan 21 14:14:18 repdb006 kernel: NFS: nfs_do_filldir() filling ended @ cookie 3075; returning = 0 Jan 21 14:14:18 repdb006 kernel: NFS: readdir_search_pagecache() searching for cookie 3075 Jan 21 14:14:18 repdb006 kernel: NFS: find_dirent_page() searching directory page 4 Jan 21 14:14:18 repdb006 kernel: NFS: nfs_readdir_filler() reading cookie 3075 into page 4. Jan 21 14:14:18 repdb006 kernel: NFS call readdirplus 3075 Jan 21 14:14:18 repdb006 kernel: NFS: nfs_update_inode(0:13/-1009531860 ct=1 info=0x6) Jan 21 14:14:18 repdb006 kernel: NFS reply readdir: 2 Jan 21 14:14:18 repdb006 kernel: NFS: found cookie 118 Jan 21 14:14:18 repdb006 kernel: NFS: find_dirent() returns 0 Jan 21 14:14:18 repdb006 kernel: NFS: find_dirent_page() returns 0 Jan 21 14:14:18 repdb006 kernel: NFS: readdir_search_pagecache() returned 0 Jan 21 14:14:18 repdb006 kernel: NFS: nfs_do_filldir() filling starting @ cookie 118 Jan 21 14:14:18 repdb006 kernel: NFS: dentry_delete(full/test, 8) Jan 21 14:14:18 repdb006 kernel: NFS: nfs_do_filldir() filling ended @ cookie 3076; returning = 0 Jan 21 14:14:18 repdb006 kernel: NFS: readdir_search_pagecache() searching for cookie 3076 Jan 21 14:14:18 repdb006 kernel: NFS: find_dirent_page() searching directory page 5 Jan 21 14:14:18 repdb006 kernel: NFS: nfs_readdir_filler() reading cookie 3076 into page 5. Jan 21 14:14:18 repdb006 kernel: NFS call readdirplus 3076 Jan 21 14:14:19 repdb006 kernel: NFS reply readdir: 0 Jan 21 14:14:19 repdb006 kernel: NFS: find_dirent() returns -523 Jan 21 14:14:19 repdb006 kernel: NFS: find_dirent_page() returns -523 Jan 21 14:14:19 repdb006 kernel: NFS: readdir_search_pagecache() returned -523 Jan 21 14:14:19 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:14:19 repdb006 kernel: NFS: revalidating (0:13/-5879706972258738325) Jan 21 14:14:19 repdb006 kernel: NFS call getattr Jan 21 14:14:20 repdb006 kernel: NFS reply getattr Jan 21 14:14:20 repdb006 kernel: NFS: nfs_update_inode(0:13/-1991882046 ct=1 info=0x6) Jan 21 14:14:20 repdb006 kernel: NFS: (0:13/-5879706972258738325) revalidation complete Jan 21 14:14:20 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz.11, 8) Jan 21 14:14:20 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:14:20 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/-1991882046, 32768) = 00000000 Jan 21 14:14:20 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz.11, 8) Jan 21 14:14:20 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:14:20 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/-1991882046, 16384) = 00000000 Jan 21 14:14:20 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz.11, 8) Jan 21 14:14:20 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:14:20 repdb006 kernel: NFS: revalidating (0:13/-8660661468576868760) Jan 21 14:14:20 repdb006 kernel: NFS call getattr Jan 21 14:14:21 repdb006 kernel: NFS reply getattr Jan 21 14:14:21 repdb006 kernel: NFS: nfs_update_inode(0:13/1247752239 ct=1 info=0x6) Jan 21 14:14:21 repdb006 kernel: NFS: (0:13/-8660661468576868760) revalidation complete Jan 21 14:14:21 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz.1, 8) Jan 21 14:14:21 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:14:21 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/1247752239, 32768) = 00000000 Jan 21 14:14:21 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz.1, 8) Jan 21 14:14:21 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:14:21 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/1247752239, 16384) = 00000000 Jan 21 14:14:21 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz.1, 8) Jan 21 14:14:21 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:14:21 repdb006 kernel: NFS: revalidating (0:13/-8417256840809953812) Jan 21 14:14:21 repdb006 kernel: NFS call getattr Jan 21 14:14:22 repdb006 kernel: NFS reply getattr Jan 21 14:14:22 repdb006 kernel: NFS: nfs_update_inode(0:13/2137268371 ct=1 info=0x6) Jan 21 14:14:22 repdb006 kernel: NFS: (0:13/-8417256840809953812) revalidation complete Jan 21 14:14:22 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz, 8) Jan 21 14:14:22 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:14:22 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/2137268371, 32768) = 00000000 Jan 21 14:14:22 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz, 8) Jan 21 14:14:22 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:14:22 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/2137268371, 16384) = 00000000 Jan 21 14:14:22 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz, 8) Jan 21 14:14:22 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:14:22 repdb006 kernel: NFS: revalidating (0:13/-5992023206934155491) Jan 21 14:14:22 repdb006 kernel: NFS call getattr Jan 21 14:14:23 repdb006 kernel: NFS reply getattr Jan 21 14:14:23 repdb006 kernel: NFS: nfs_update_inode(0:13/1544741310 ct=1 info=0x6) Jan 21 14:14:23 repdb006 kernel: NFS: (0:13/-5992023206934155491) revalidation complete Jan 21 14:14:23 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz.111, 8) Jan 21 14:14:23 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:14:23 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/1544741310, 32768) = 00000000 Jan 21 14:14:23 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz.111, 8) Jan 21 14:14:23 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:14:23 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/1544741310, 16384) = 00000000 Jan 21 14:14:23 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz.111, 8) Jan 21 14:14:23 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:14:23 repdb006 kernel: NFS: revalidating (0:13/-5887948837043818545) Jan 21 14:14:23 repdb006 kernel: NFS call getattr Jan 21 14:14:24 repdb006 kernel: NFS reply getattr Jan 21 14:14:24 repdb006 kernel: NFS: nfs_update_inode(0:13/-702684555 ct=1 info=0x6) Jan 21 14:14:24 repdb006 kernel: NFS: (0:13/-5887948837043818545) revalidation complete Jan 21 14:14:24 repdb006 kernel: NFS: dentry_delete(full/test, 8) Jan 21 14:14:24 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:14:24 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/-702684555, 32768) = 00000000 Jan 21 14:14:24 repdb006 kernel: NFS: dentry_delete(full/test, 8) Jan 21 14:14:24 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:14:24 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/-702684555, 16384) = 00000000 Jan 21 14:14:24 repdb006 kernel: NFS: dentry_delete(full/test, 8) Jan 21 14:14:24 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:14:24 repdb006 kernel: NFS: revalidating (0:13/-8936416646988302039) Jan 21 14:14:24 repdb006 kernel: NFS call getattr Jan 21 14:14:25 repdb006 kernel: NFS reply getattr Jan 21 14:14:25 repdb006 kernel: NFS: nfs_update_inode(0:13/-1666940607 ct=1 info=0x6) Jan 21 14:14:25 repdb006 kernel: NFS: (0:13/-8936416646988302039) revalidation complete Jan 21 14:14:25 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:14:25 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/-1666940607, 32768) = fffffff5 Jan 21 14:14:25 repdb006 kernel: NFS call getacl Jan 21 14:14:25 repdb006 kernel: NFS reply getacl: 0 Jan 21 14:14:26 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:14:26 repdb006 kernel: NFS: revalidating (0:13/-8936416646988302039) Jan 21 14:14:26 repdb006 kernel: NFS call getattr Jan 21 14:14:26 repdb006 kernel: NFS reply getattr Jan 21 14:14:26 repdb006 kernel: NFS: nfs_update_inode(0:13/-1666940607 ct=1 info=0x6) Jan 21 14:14:26 repdb006 kernel: NFS: (0:13/-8936416646988302039) revalidation complete Jan 21 14:14:26 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/-1666940607, 16384) = 00000000 Jan 21 14:14:26 repdb006 kernel: NFS: readdir_search_pagecache() searching for cookie 3076 Jan 21 14:14:26 repdb006 kernel: NFS: find_dirent_page() searching directory page 0 Jan 21 14:14:26 repdb006 kernel: NFS: found cookie 25 Jan 21 14:14:26 repdb006 kernel: NFS: found cookie 37 Jan 21 14:14:27 repdb006 kernel: NFS: find_dirent() returns -11 Jan 21 14:14:27 repdb006 kernel: NFS: find_dirent_page() returns -11 Jan 21 14:14:27 repdb006 kernel: NFS: find_dirent_page() searching directory page 1 Jan 21 14:14:27 repdb006 kernel: NFS: found cookie 3073 Jan 21 14:14:27 repdb006 kernel: NFS: find_dirent() returns -11 Jan 21 14:14:27 repdb006 kernel: NFS: find_dirent_page() returns -11 Jan 21 14:14:27 repdb006 kernel: NFS: find_dirent_page() searching directory page 2 Jan 21 14:14:27 repdb006 kernel: NFS: found cookie 3074 Jan 21 14:14:27 repdb006 kernel: NFS: find_dirent() returns -11 Jan 21 14:14:27 repdb006 kernel: NFS: find_dirent_page() returns -11 Jan 21 14:14:27 repdb006 kernel: NFS: find_dirent_page() searching directory page 3 Jan 21 14:14:27 repdb006 kernel: NFS: found cookie 63 Jan 21 14:14:27 repdb006 kernel: NFS: found cookie 3075 Jan 21 14:14:27 repdb006 kernel: NFS: find_dirent() returns -11 Jan 21 14:14:27 repdb006 kernel: NFS: find_dirent_page() returns -11 Jan 21 14:14:27 repdb006 kernel: NFS: find_dirent_page() searching directory page 4 Jan 21 14:14:27 repdb006 kernel: NFS: found cookie 118 Jan 21 14:14:27 repdb006 kernel: NFS: found cookie 3076 Jan 21 14:14:27 repdb006 kernel: NFS: find_dirent() returns -11 Jan 21 14:14:27 repdb006 kernel: NFS: find_dirent_page() returns -11 Jan 21 14:14:27 repdb006 kernel: NFS: find_dirent_page() searching directory page 5 Jan 21 14:14:27 repdb006 kernel: NFS: find_dirent() returns -523 Jan 21 14:14:27 repdb006 kernel: NFS: find_dirent_page() returns -523 Jan 21 14:14:27 repdb006 kernel: NFS: readdir_search_pagecache() returned -523 Jan 21 14:15:16 repdb006 kernel: NFS: revalidating (0:13/-8936416646988302039) Jan 21 14:15:16 repdb006 kernel: NFS call getattr Jan 21 14:15:17 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz.2, 8) Jan 21 14:15:28 repdb006 kernel: NFS reply getattr Jan 21 14:15:28 repdb006 kernel: NFS: nfs_update_inode(0:13/-702684555 ct=1 info=0x6) Jan 21 14:15:29 repdb006 kernel: NFS: (0:13/-5887948837043818545) revalidation complete Jan 21 14:15:29 repdb006 kernel: NFS: dentry_delete(full/test, 8) Jan 21 14:15:29 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:15:29 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/-702684555, 32768) = 00000000 Jan 21 14:15:29 repdb006 kernel: NFS: dentry_delete(full/test, 8) Jan 21 14:15:29 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:15:29 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/-702684555, 16384) = 00000000 Jan 21 14:15:29 repdb006 kernel: NFS: dentry_delete(full/test, 8) Jan 21 14:15:29 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:15:29 repdb006 kernel: NFS: revalidating (0:13/-8936416646988302039) Jan 21 14:15:29 repdb006 kernel: NFS call getattr Jan 21 14:15:29 repdb006 kernel: NFS reply getattr Jan 21 14:15:29 repdb006 kernel: NFS: nfs_update_inode(0:13/-1666940607 ct=1 info=0x6) Jan 21 14:15:30 repdb006 kernel: NFS: (0:13/-8936416646988302039) revalidation complete Jan 21 14:15:30 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz.2, 8) Jan 21 14:15:30 repdb006 kernel: NFS: permission(0:13ry page 3 Jan 21 14:15:30 repdb006 kernel: NFS: found cookie 63 Jan 21 14:15:30 repdb006 kernel: NFS: found cookie 3075 Jan 21 14:15:30 repdb006 kernel: NFS: find_dirent() returns -11 Jan 21 14:15:30 repdb006 kernel: NFS: find_dirent_page() returns -11 Jan 21 14:15:30 repdb006 kernel: NFS: find_dirent_page() searching directory page 4 Jan 21 14:15:30 repdb006 kernel: NFS: found cookie 118 Jan 21 14:15:30 repdb006 kernel: NFS: found cookie 3076 Jan 21 14:15:30 repdb006 kernel: NFS: find_dirent() returns -11 Jan 21 14:15:30 repdb006 kernel: NFS: find_dirent_page() returns -11 Jan 21 14:15:30 repdb006 kernel: NFS: find_dirent_page() searching directory page 5 Jan 21 14:15:30 repdb006 kernel: NFS: find_dirent() returns -523 Jan 21 14:15:30 repdb006 kernel: NFS: find_dirent_page() returns -523 Jan 21 14:15:30 repdb006 kernel: NFS: readdir_search_pagecache() returned -523 Jan 21 14:15:37 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:15:37 repdb006 kernel: NFS: revalidating (0:13/-6148448765687654848) Jan 21 14:15:37 repdb006 kernel: NFS call getattr Jan 21 14:15:38 repdb006 kernel: NFS reply getattr Jan 21 14:15:38 repdb006 kernel: NFS: nfs_update_inode(0:13/-1009531860 ct=1 info=0x6) Jan 21 14:15:38 repdb006 kernel: NFS: (0:13/-6148448765687654848) revalidation complete Jan 21 14:15:38 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x4, res=0 Jan 21 14:15:40 repdb006 kernel: NFS reply lookup: 0 Jan 21 14:15:40 repdb006 kernel: NFS: nfs_update_inode(0:13/1247752239 ct=1 info=0x6) Jan 21 14:15:40 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz.1, 8) Jan 21 14:15:40 repdb006 kernel: NFS call access Jan 21 14:15:41 repdb006 kernel: NFS reply access, status = 0 Jan 21 14:15:41 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:15:41 repdb006 kernel: NFS: revalidating (0:13/-6148448765687654848) Jan 21 14:15:41 repdb006 kernel: NFS call getattr Jan 21 14:15:41 repdb006 kernel: NFS reply getattr Jan 21 14:15:41 repdb006 kernel: NFS: nfs_update_inode(0:13/-1009531860 ct=1 info=0x6) Jan 21 14:15:41 repdb006 kernel: NFS: (0:13/-6148448765687654848) revalidation complete Jan 21 14:15:41 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz.1, 8) Jan 21 14:15:41 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:15:41 repdb006 kernel: NFS call access Jan 21 14:15:42 repdb006 kernel: NFS reply access, status = 0 Jan 21 14:15:42 repdb006 kernel: NFS: permission(0:13/1247752239), mask=0x2, res=0 Jan 21 14:15:42 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz.1, 8) Jan 21 14:15:43 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:15:43 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:15:43 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x3, res=0 Jan 21 14:15:43 repdb006 kernel: NFS: unlink(0:13/-1009531860, pr_rssd_dss_master.jz.1) Jan 21 14:15:43 repdb006 kernel: NFS: safe_remove(full/pr_rssd_dss_master.jz.1) Jan 21 14:15:43 repdb006 kernel: NFS call remove pr_rssd_dss_master.jz.1 Jan 21 14:15:44 repdb006 kernel: NFS: mtime change on server for file 0:13/-1009531860 Jan 21 14:15:44 repdb006 kernel: NFS reply remove: 0 Jan 21 14:15:44 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz.1, 18) Jan 21 14:15:44 repdb006 kernel: NFS: delete_inode(0:13/1247752239) Jan 21 14:15:44 repdb006 kernel: NFS: nfs3_forget_cached_acls(0:13/1247752239) Jan 21 14:15:44 repdb006 kernel: NFS call access Jan 21 14:15:45 repdb006 kernel: NFS reply access, status = 0 Jan 21 14:15:45 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:15:45 repdb006 kernel: NFS: revalidating (0:13/-6148448765687654848) Jan 21 14:15:45 repdb006 kernel: NFS call getattr Jan 21 14:15:46 repdb006 kernel: NFS reply lookup: 0 Jan 21 14:15:46 repdb006 kernel: NFS: nfs_update_inode(0:13/-1009531860 ct=1 info=0x6) Jan 21 14:15:46 repdb006 kernel: NFS: isize change on server for file 0:13/-1009531860 Jan 21 14:15:46 repdb006 kernel: NFS: mtime change on server for file 0:13/-1009531860 Jan 21 14:15:46 repdb006 kernel: NFS: nfs_update_inode(0:13/-1991882046 ct=1 info=0x6) Jan 21 14:15:46 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz.11, 8) Jan 21 14:15:46 repdb006 kernel: NFS call access Jan 21 14:15:47 repdb006 kernel: NFS reply getattr Jan 21 14:15:47 repdb006 kernel: NFS: nfs_update_inode(0:13/-1009531860 ct=1 info=0x6) Jan 21 14:15:47 repdb006 kernel: NFS: isize change on server for file 0:13/-1009531860 Jan 21 14:15:47 repdb006 kernel: NFS: mtime change on server for file 0:13/-1009531860 Jan 21 14:15:47 repdb006 kernel: NFS: (0:13/-6148448765687654848) data cache invalidated Jan 21 14:15:47 repdb006 kernel: NFS: nfs3_forget_cached_acls(0:13/-1009531860) Jan 21 14:15:47 repdb006 kernel: NFS: (0:13/-6148448765687654848) revalidation complete Jan 21 14:15:47 repdb006 kernel: NFS call lookup pr_rssd_dss_master.jz.11 Jan 21 14:15:49 repdb006 kernel: NFS reply access, status = 0 Jan 21 14:15:49 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:15:49 repdb006 kernel: NFS: revalidating (0:13/-6148448765687654848) Jan 21 14:15:49 repdb006 kernel: NFS call getattr Jan 21 14:15:50 repdb006 kernel: NFS reply getattr Jan 21 14:15:50 repdb006 kernel: NFS: nfs_update_inode(0:13/-1009531860 ct=1 info=0x6) Jan 21 14:15:50 repdb006 kernel: NFS: (0:13/-6148448765687654848) revalidation complete Jan 21 14:15:50 repdb006 kernel: NFS call access Jan 21 14:15:50 repdb006 kernel: NFS reply access, status = 0 Jan 21 14:15:50 repdb006 kernel: NFS: permission(0:13/-1991882046), mask=0x2, res=0 Jan 21 14:15:50 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz.11, 8) Jan 21 14:15:51 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:15:52 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:15:52 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x3, res=0 Jan 21 14:15:52 repdb006 kernel: NFS: unlink(0:13/-1009531860, pr_rssd_dss_master.jz.11) Jan 21 14:15:52 repdb006 kernel: NFS: safe_remove(full/pr_rssd_dss_master.jz.11) Jan 21 14:15:52 repdb006 kernel: NFS call remove pr_rssd_dss_master.jz.11 Jan 21 14:15:52 repdb006 kernel: NFS: nfs_update_inode(0:13/-1009531860 ct=1 info=0x7) Jan 21 14:15:52 repdb006 kernel: NFS: isize change on server for file 0:13/-1009531860 Jan 21 14:15:52 repdb006 kernel: NFS: mtime change on server for file 0:13/-1009531860 Jan 21 14:15:52 repdb006 kernel: NFS reply remove: 0 Jan 21 14:15:52 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz.11, 18) Jan 21 14:15:52 repdb006 kernel: NFS: delete_inode(0:13/-1991882046) Jan 21 14:15:53 repdb006 kernel: NFS: nfs3_forget_cached_acls(0:13/-1991882046) Jan 21 14:15:53 repdb006 kernel: NFS call access Jan 21 14:15:53 repdb006 kernel: NFS reply access, status = 0 Jan 21 14:15:53 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:15:53 repdb006 kernel: NFS: revalidating (0:13/-6148448765687654848) Jan 21 14:15:53 repdb006 kernel: NFS call getattr Jan 21 14:15:54 repdb006 kernel: NFS reply getattr Jan 21 14:15:54 repdb006 kernel: NFS: nfs_update_inode(0:13/-1009531860 ct=1 info=0x6) Jan 21 14:15:54 repdb006 kernel: NFS: isize change on server for file 0:13/-1009531860 Jan 21 14:15:54 repdb006 kernel: NFS: mtime change on server for file 0:13/-1009531860 Jan 21 14:15:54 repdb006 kernel: NFS: (0:13/-6148448765687654848) data cache invalidated Jan 21 14:15:54 repdb006 kernel: NFS: nfs3_forget_cached_acls(0:13/-1009531860) Jan 21 14:15:54 repdb006 kernel: NFS: (0:13/-6148448765687654848) revalidation complete Jan 21 14:15:54 repdb006 kernel: NFS call lookup pr_rssd_dss_master.jz.111 Jan 21 14:15:54 repdb006 kernel: NFS reply lookup: 0 Jan 21 14:15:54 repdb006 kernel: NFS: nfs_update_inode(0:13/1544741310 ct=1 info=0x6) Jan 21 14:15:54 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz.111, 8) Jan 21 14:15:54 repdb006 kernel: NFS call access Jan 21 14:15:55 repdb006 kernel: NFS reply access, status = 0 Jan 21 14:15:55 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:15:55 repdb006 kernel: NFS: revalidating (0:13/-6148448765687654848) Jan 21 14:15:55 repdb006 kernel: NFS call getattr Jan 21 14:15:56 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:15:56 repdb006 kernel: NFS call access Jan 21 14:15:56 repdb006 kernel: NFS reply access, status = 0 Jan 21 14:15:57 repdb006 kernel: NFS: permission(0:13/1544741310), mask=0x2, res=0 Jan 21 14:15:57 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz.111, 8) Jan 21 14:15:57 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:15:57 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:15:57 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x3, res=0 Jan 21 14:15:57 repdb006 kernel: NFS: unlink(0:13/-1009531860, pr_rssd_dss_master.jz.111) Jan 21 14:15:57 repdb006 kernel: NFS: safe_remove(full/pr_rssd_dss_master.jz.111) Jan 21 14:15:57 repdb006 kernel: NFS call remove pr_rssd_dss_master.jz.111 Jan 21 14:15:57 repdb006 kernel: NFS: delete_inode(0:13/1544741310) Jan 21 14:15:57 repdb006 kernel: NFS: nfs3_forget_cached_acls(0:13/1544741310) Jan 21 14:15:57 repdb006 kernel: NFS call access Jan 21 14:15:58 repdb006 kernel: NFS reply access, status = 0 Jan 21 14:15:58 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:15:58 repdb006 kernel: NFS: revalidating (0:13/-6148448765687654848) Jan 21 14:15:58 repdb006 kernel: NFS call getattr Jan 21 14:15:59 repdb006 kernel: NFS reply lookup: 0 Jan 21 14:15:59 repdb006 kernel: NFS: nfs_update_inode(0:13/-1009531860 ct=1 info=0x6) Jan 21 14:15:59 repdb006 kernel: NFS: isize change on server for file 0:13/-1009531860 Jan 21 14:15:59 repdb006 kernel: NFS: mtime change on server for file 0:13/-1009531860 Jan 21 14:15:59 repdb006 kernel: NFS: nfs_update_inode(0:13/-1666940607 ct=1 info=0x6) Jan 21 14:15:59 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz.2, 8) Jan 21 14:15:59 repdb006 kernel: NFS call access Jan 21 14:16:00 repdb006 kernel: NFS reply access, status = 0 Jan 21 14:16:00 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:16:00 repdb006 kernel: NFS: revalidating (0:13/-6148448765687654848) Jan 21 14:16:00 repdb006 kernel: NFS call getattr Jan 21 14:16:01 repdb006 kernel: NFS reply getattr Jan 21 14:16:01 repdb006 kernel: NFS: nfs_update_inode(0:13/-1009531860 ct=1 info=0x6) Jan 21 14:16:01 repdb006 kernel: NFS: (0:13/-6148448765687654848) revalidation complete Jan 21 14:16:01 repdb006 kernel: NFS call access Jan 21 14:16:01 repdb006 kernel: NFS reply access, status = 0 Jan 21 14:16:01 repdb006 kernel: NFS: permission(0:13/-1666940607), mask=0x2, res=0 Jan 21 14:16:01 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz.2, 8) Jan 21 14:16:01 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:16:01 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:16:02 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x3, res=0 Jan 21 14:16:02 repdb006 kernel: NFS: unlink(0:13/-1009531860, pr_rssd_dss_master.jz.2) Jan 21 14:16:02 repdb006 kernel: NFS: safe_remove(full/pr_rssd_dss_master.jz.2) Jan 21 14:16:02 repdb006 kernel: NFS call remove pr_rssd_dss_master.jz.2 Jan 21 14:16:02 repdb006 kernel: NFS: nfs_update_inode(0:13/-1009531860 ct=1 info=0x7) Jan 21 14:16:02 repdb006 kernel: NFS: isize change on server for file 0:13/-1009531860 Jan 21 14:16:02 repdb006 kernel: NFS: mtime change on server for file 0:13/-1009531860 Jan 21 14:16:02 repdb006 kernel: NFS reply remove: 0 Jan 21 14:16:02 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz.2, 18) Jan 21 14:16:02 repdb006 kernel: NFS: delete_inode(0:13/-1666940607) Jan 21 14:16:02 repdb006 kernel: NFS: nfs3_forget_cached_acls(0:13/-1666940607) Jan 21 14:16:03 repdb006 kernel: NFS call access Jan 21 14:16:03 repdb006 kernel: NFS reply access, status = 0 Jan 21 14:16:03 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:16:03 repdb006 kernel: NFS: revalidating (0:13/-6148448765687654848) Jan 21 14:16:03 repdb006 kernel: NFS call getattr Jan 21 14:16:04 repdb006 kernel: NFS reply getattr Jan 21 14:16:04 repdb006 kernel: NFS: nfs_update_inode(0:13/-1009531860 ct=1 info=0x6) Jan 21 14:16:04 repdb006 kernel: NFS: isize change on server for file 0:13/-1009531860 Jan 21 14:16:04 repdb006 kernel: NFS: mtime change on server for file 0:13/-1009531860 Jan 21 14:16:04 repdb006 kernel: NFS: (0:13/-6148448765687654848) data cache invalidated Jan 21 14:16:04 repdb006 kernel: NFS: nfs3_forget_cached_acls(0:13/-1009531860) Jan 21 14:16:04 repdb006 kernel: NFS: (0:13/-6148448765687654848) revalidation complete Jan 21 14:16:04 repdb006 kernel: NFS call lookup test Jan 21 14:16:05 repdb006 kernel: NFS reply access, status = 0 Jan 21 14:16:05 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:16:05 repdb006 kernel: NFS: revalidating (0:13/-6148448765687654848) Jan 21 14:16:05 repdb006 kernel: NFS call getattr Jan 21 14:16:06 repdb006 kernel: NFS reply access, status = 0 Jan 21 14:16:06 repdb006 kernel: NFS: permission(0:13/-702684555), mask=0x2, res=0 Jan 21 14:16:06 repdb006 kernel: NFS: dentry_delete(full/test, 8) Jan 21 14:16:06 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:16:07 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:16:07 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x3, res=0 Jan 21 14:16:07 repdb006 kernel: NFS: unlink(0:13/-1009531860, test) Jan 21 14:16:07 repdb006 kernel: NFS: safe_remove(full/test) Jan 21 14:16:07 repdb006 kernel: NFS call remove test Jan 21 14:16:07 repdb006 kernel: NFS: nfs_update_inode(0:13/-1009531860 ct=1 info=0x7) Jan 21 14:16:08 repdb006 kernel: NFS reply access, status = 0 Jan 21 14:16:08 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:16:08 repdb006 kernel: NFS: revalidating (0:13/-6148448765687654848) Jan 21 14:16:08 repdb006 kernel: NFS call getattr Jan 21 14:16:09 repdb006 kernel: NFS reply getattr Jan 21 14:16:09 repdb006 kernel: NFS: nfs_update_inode(0:13/-1009531860 ct=1 info=0x6) Jan 21 14:16:09 repdb006 kernel: NFS: isize change on server for file 0:13/-1009531860 Jan 21 14:16:09 repdb006 kernel: NFS: mtime change on server for file 0:13/-1009531860 Jan 21 14:16:09 repdb006 kernel: NFS: (0:13/-6148448765687654848) data cache invalidated Jan 21 14:16:09 repdb006 kernel: NFS: nfs3_forget_cached_acls(0:13/-1009531860) Jan 21 14:16:09 repdb006 kernel: NFS: (0:13/-6148448765687654848) revalidation complete Jan 21 14:16:09 repdb006 kernel: NFS call access Jan 21 14:16:09 repdb006 kernel: NFS reply access, status = 0 Jan 21 14:16:09 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:16:09 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:16:09 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:16:09 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:16:09 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:16:11 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:16:11 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:16:11 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:16:11 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:16:11 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:16:11 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:16:12 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:16:12 repdb006 kernel: NFS: revalidating (0:13/-6148448765687654848) Jan 21 14:16:12 repdb006 kernel: NFS call getattr Jan 21 14:16:13 repdb006 kernel: NFS reply getattr Jan 21 14:16:13 repdb006 kernel: NFS: nfs_update_inode(0:13/-1009531860 ct=1 info=0x6) Jan 21 14:16:13 repdb006 kernel: NFS: (0:13/-6148448765687654848) revalidation complete Jan 21 14:16:13 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x4, res=0 Jan 21 14:16:13 repdb006 kernel: NFS: readdir_search_pagecache() searching for offset 0 Jan 21 14:16:13 repdb006 kernel: NFS: find_dirent_page() searching directory page 0 Jan 21 14:16:13 repdb006 kernel: NFS: nfs_readdir_filler() reading cookie 0 into page 0. Jan 21 14:16:13 repdb006 kernel: NFS call readdirplus 0 Jan 21 14:16:14 repdb006 kernel: NFS reply readdir: 2 Jan 21 14:16:14 repdb006 kernel: NFS: found cookie 25 at index 0 Jan 21 14:16:14 repdb006 kernel: NFS: find_dirent_index() returns 0 Jan 21 14:16:14 repdb006 kernel: NFS: find_dirent_page() returns 0 Jan 21 14:16:14 repdb006 kernel: NFS: readdir_search_pagecache() returned 0 Jan 21 14:16:14 repdb006 kernel: NFS: nfs_do_filldir() filling starting @ cookie 25 Jan 21 14:16:14 repdb006 kernel: NFS: nfs_do_filldir() filling ended @ cookie 3073; returning = 0 Jan 21 14:16:14 repdb006 kernel: NFS: readdir_search_pagecache() searching for cookie 3073 Jan 21 14:16:14 repdb006 kernel: NFS: find_dirent_page() searching directory page 1 Jan 21 14:16:14 repdb006 kernel: NFS: nfs_readdir_filler() reading cookie 3073 into page 1. Jan 21 14:16:14 repdb006 kernel: NFS call readdirplus 3073 Jan 21 14:16:15 repdb006 kernel: NFS reply getacl: 0 Jan 21 14:16:15 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:16:15 repdb006 kernel: NFS: revalidating (0:13/-6148448765687654848) Jan 21 14:16:15 repdb006 kernel: NFS call getattr Jan 21 14:16:16 repdb006 kernel: NFS reply getattr Jan 21 14:16:16 repdb006 kernel: NFS: nfs_update_inode(0:13/-1009531860 ct=1 info=0x6) Jan 21 14:16:16 repdb006 kernel: NFS: (0:13/-6148448765687654848) revalidation complete Jan 21 14:16:16 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/-1009531860, 16384) = 00000000 Jan 21 14:16:16 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:16:16 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:16:16 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/487151148, 32768) = 00000000 Jan 21 14:16:16 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:16:16 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/487151148, 16384) = 00000000 Jan 21 14:16:16 repdb006 kernel: NFS: readdir_search_pagecache() searching for cookie 3073 Jan 21 14:16:16 repdb006 kernel: NFS: find_dirent_page() searching directory page 0 Jan 21 14:16:16 repdb006 kernel: NFS: found cookie 25 Jan 21 14:16:16 repdb006 kernel: NFS: found cookie 3073 Jan 21 14:16:16 repdb006 kernel: NFS: find_dirent() returns -11 Jan 21 14:16:16 repdb006 kernel: NFS: find_dirent_page() returns -11 Jan 21 14:16:16 repdb006 kernel: NFS: find_dirent_page() searching directory page 1 Jan 21 14:16:16 repdb006 kernel: NFS: find_dirent() returns -523 Jan 21 14:16:16 repdb006 kernel: NFS: find_dirent_page() returns -523 Jan 21 14:16:16 repdb006 kernel: NFS: readdir_search_pagecache() returned -523 Jan 21 14:16:16 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:16:16 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:16:16 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:16:16 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:16:16 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:16:16 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:16:33 repdb006 kernel: NFS reply access, status = 0 Jan 21 14:16:33 repdb006 kernel: NFS: permission(0:13/-1964955375), mask=0x1, res=0 Jan 21 14:16:33 repdb006 kernel: NFS call access Jan 21 14:16:34 repdb006 kernel: NFS reply access, status = 0 Jan 21 14:16:34 repdb006 kernel: NFS: permission(0:13/487151148), mask=0x1, res=0 Jan 21 14:16:34 repdb006 kernel: NFS: permission(0:13/-10091630 call_encode (status 0) Jan 21 14:16:35 repdb006 kernel: NFS reply getattr Jan 21 14:16:35 repdb006 kernel: NFS: nfs_update_inode(0:13/-1009531860 ct=1 info=0x6) Jan 21 14:16:35 repdb006 kernel: NFS: isize change on server for file 0:13/-1009531860 Jan 21 14:16:35 repdb006 kernel: NFS: mtime change on server for file 0:13/-1009531860 Jan 21 14:16:35 repdb006 kernel: NFS: (0:13/-6148448765687654848) data cache invalidated Jan 21 14:16:35 repdb006 kernel: NFS: nfs3_forget_cached_acls(0:13/-1009531860) Jan 21 14:16:35 repdb006 kernel: NFS: (0:13/-6148448765687654848) revalidation complete Jan 21 14:16:35 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/-1009531860, 16384) = fffffff5 Jan 21 14:16:35 repdb006 kernel: NFS call getacl Jan 21 14:16:36 repdb006 kernel: NFS reply getacl: 0 Jan 21 14:16:36 repdb006 kernel: NFS reply getattr Jan 21 14:16:36 repdb006 kernel: NFS: nfs_update_inode(0:13/-1009531860 ct=1 info=0x6) Jan 21 14:16:36 repdb006 kernel: NFS: isize change on server for file 0:13/-1009531860 Jan 21 14:16:36 repdb006 kernel: NFS: (0:13/-6148448765687654848) data cache invalidated Jan 21 14:16:36 repdb006 kernel: NFS: (0:13/-6148448765687654848) revalidation complete Jan 21 14:16:36 repdb006 kernel: NFS call access Jan 21 14:16:37 repdb006 kernel: NFS reply access, status = 0 Jan 21 14:16:37 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:16:37 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:16:38 repdb006 kernel: NFS reply setattr Jan 21 14:16:38 repdb006 kernel: NFS: nfs_update_inode(0:13/-1093108514 ct=1 info=0x7) Jan 21 14:16:38 repdb006 kernel: NFS: nfs_lock(f=0:13/-1093108514, t=1, fl=1, r=0:9223372036854775807) Jan 21 14:16:42 repdb006 kernel: NFS call access Jan 21 14:16:42 repdb006 kernel: NFS reply access, status = 0 Jan 21 14:16:42 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:16:42 repdb006 kernel: NFS: revalidating (0:13/-6148448765687654848) Jan 21 14:16:42 repdb006 kernel: NFS call getattr Jan 21 14:16:43 repdb006 kernel: NFS: nfs3_forget_cached_acls(0:13/-1009531860) Jan 21 14:16:43 repdb006 kernel: NFS: (0:13/-6148448765687654848) revalidation complete Jan 21 14:16:43 repdb006 kernel: NFS call access Jan 21 14:16:44 repdb006 kernel: NFS reply access, status = 0 Jan 21 14:16:44 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x4, res=0 Jan 21 14:16:44 repdb006 kernel: NFS: readdir_search_pagecache() searching for offset 0 Jan 21 14:16:44 repdb006 kernel: NFS: find_dirent_page() searching directory page 0 Jan 21 14:16:44 repdb006 kernel: NFS: nfs_readdir_filler() reading cookie 0 into page 0. Jan 21 14:16:44 repdb006 kernel: NFS call readdirplus 0 Jan 21 14:16:45 repdb006 kernel: NFS: nfs_update_inode(0:13/-1009531860 ct=1 info=0x6) Jan 21 14:16:45 repdb006 kernel: NFS reply readdir: 0 Jan 21 14:16:45 repdb006 kernel: NFS: find_dirent() returns -523 Jan 21 14:16:45 repdb006 kernel: NFS: find_dirent_page() returns -523 Jan 21 14:16:45 repdb006 kernel: NFS: readdir_search_pagecache() returned -523 Jan 21 14:16:46 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:16:46 repdb006 kernel: NFS: revalidating (0:13/-5731948363835999349) Jan 21 14:16:46 repdb006 kernel: NFS call getattr Jan 21 14:16:47 repdb006 kernel: NFS reply getattr Jan 21 14:16:47 repdb006 kernel: NFS: nfs_update_inode(0:13/-1093108514 ct=1 info=0x6) Jan 21 14:16:47 repdb006 kernel: NFS: (0:13/-5731948363835999349) revalidation complete Jan 21 14:16:47 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:16:47 repdb006 kernel: NFS: nfs3_get_cached_acl(0:13/-1093108514, 32768) = fffffff5 Jan 21 14:16:47 repdb006 kernel: NFS call getacl Jan 21 14:16:55 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:16:55 repdb006 kernel: NFS: revalidating (0:13/-8058565423364083875) Jan 21 14:16:55 repdb006 kernel: NFS call getattr Jan 21 14:16:56 repdb006 kernel: NFS reply getattr Jan 21 14:16:56 repdb006 kernel: NFS: nfs_update_inode(0:13/487151148 ct=1 info=0x6) Jan 21 14:16:56 repdb006 kernel: NFS: nfs3_forget_cached_acls(0:13/487151148) Jan 21 14:16:56 repdb006 kernel: NFS: (0:13/-8058565423364083875) revalidation complete Jan 21 14:16:56 repdb006 kernel: NFS: revalidating (0:13/-6148448765687654848) Jan 21 14:16:56 repdb006 kernel: NFS call getattr Jan 21 14:16:56 repdb006 kernel: NFS reply getattr Jan 21 14:16:56 repdb006 kernel: NFS: nfs_update_inode(0:13/-1009531860 ct=1 info=0x6) Jan 21 14:16:56 repdb006 kernel: NFS: (0:13/-6148448765687654848) revalidation complete Jan 21 14:16:57 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x4, res=0 Jan 21 14:16:57 repdb006 kernel: NFS: readdir_search_pagecache() searching for offset 0 Jan 21 14:16:57 repdb006 kernel: NFS: find_dirent_page() searching directory page 0 Jan 21 14:16:57 repdb006 kernel: NFS: found cookie 25 at index 0 Jan 21 14:16:57 repdb006 kernel: NFS: find_dirent_index() returns 0 Jan 21 14:16:57 repdb006 kernel: NFS: find_dirent_page() returns 0 Jan 21 14:16:57 repdb006 kernel: NFS: readdir_search_pagecache() returned 0 Jan 21 14:16:57 repdb006 kernel: NFS: nfs_do_filldir() filling starting @ cookie 25 Jan 21 14:16:57 repdb006 kernel: NFS: nfs_do_filldir() filling ended @ cookie 3073; returning = 0 Jan 21 14:16:57 repdb006 kernel: NFS: readdir_search_pagecache() searching for cookie 3073 Jan 21 14:16:57 repdb006 kernel: NFS: find_dirent_page() searching directory page 1 Jan 21 14:16:57 repdb006 kernel: NFS: found cookie 3075 Jan 21 14:16:57 repdb006 kernel: NFS: find_dirent() returns 0 Jan 21 14:16:57 repdb006 kernel: NFS: find_dirent_page() returns 0 Jan 21 14:16:57 repdb006 kernel: NFS: readdir_search_pagecache() returned 0 Jan 21 14:16:57 repdb006 kernel: NFS: nfs_do_filldir() filling starting @ cookie 3075 Jan 21 14:16:57 repdb006 kernel: NFS: nfs_do_filldir() filling ended @ cookie 3075; returning = 0 Jan 21 14:16:57 repdb006 kernel: NFS: readdir_search_pagecache() searching for cookie 3075 Jan 21 14:16:57 repdb006 kernel: NFS: find_dirent_page() searching directory page 2 Jan 21 14:16:57 repdb006 kernel: NFS: find_dirent() returns -523 Jan 21 14:16:57 repdb006 kernel: NFS: find_dirent_page() returns -523 Jan 21 14:16:57 repdb006 kernel: NFS: readdir_search_pagecache() returned -523 Jan 21 14:16:57 repdb006 kernel: NFS: permission(0:13/-1009531860), mask=0x1, res=0 Jan 21 14:16:57 repdb006 kernel: NFS: revalidating (0:13/-5731948363835999349) Jan 21 14:16:57 repdb006 kernel: NFS call getattr Jan 21 14:18:24 repdb006 kernel: NFS: revalidating (0:13/-5731948363835999349) Jan 21 14:18:24 repdb006 kernel: NFS call getattr Jan 21 14:18:25 repdb006 kernel: NFS reply getattr Jan 21 14:18:25 repdb006 kernel: NFS: nfs_update_inode(0:13/-1093108514 ct=1 info=0x6) Jan 21 14:18:25 repdb006 kernel: NFS: (0:13/-5731948363835999349) revalidation complete Jan 21 14:18:25 repdb006 kernel: NFS: nfs_lock(f=0:13/-1093108514, t=2, fl=1, r=0:9223372036854775807) Jan 21 14:18:25 repdb006 kernel: NFS: dentry_delete(full/pr_rssd_dss_master.jz.22, 8) ________________________________________ From: Soumya Koduri [skoduri@xxxxxxxxxx] Sent: Friday, January 23, 2015 5:14 AM To: Peter Auyeung; gluster-devel@xxxxxxxxxxx; gluster-users@xxxxxxxxxxx Subject: Re: [Gluster-devel] [Gluster-user] Sybase backup server failed to write to Gluster NFS In that case, most likely it seems to be an issue with the backup servers you are using. Maybe you can first try verifying the NFS client on that machine. Issue write fops directly on the NFS mount points used by those servers. Enable rpcdebug -> "rpcdebug -m nfs all" and check "/var/log/messages" for any errors. Thanks, Soumya On 01/22/2015 11:11 PM, Peter Auyeung wrote: > Hi Soumya, > > I was able to mount the same volume on other NFS client and do writes > > got the following nfs.log entries when write > > > > > [2015-01-22 17:39:03.528405] I [afr-self-heal-common.c:2868:afr_log_self_heal_completion_status] 0-sas02-replicate-1: metadata self heal is successfully completed, metadata self heal from source sas02-client-2 to sas02-client-3, metadata - Pending matrix: [ [ 0 0 ] [ 0 0 ] ], on /RepDBSata02 > [2015-01-22 17:39:03.529407] I [afr-self-heal-common.c:2868:afr_log_self_heal_completion_status] 0-sas02-replicate-2: metadata self heal is successfully completed, metadata self heal from source sas02-client-4 to sas02-client-5, metadata - Pending matrix: [ [ 0 0 ] [ 0 0 ] ], on /RepDBSata02 > > > Thanks > Peter > ________________________________________ > From: Soumya Koduri [skoduri@xxxxxxxxxx] > Sent: Wednesday, January 21, 2015 9:05 PM > To: Peter Auyeung; gluster-devel@xxxxxxxxxxx; gluster-users@xxxxxxxxxxx > Subject: Re: [Gluster-devel] [Gluster-user] Sybase backup server failed to write to Gluster NFS > > Hi Peter, > > Can you please try manually mounting those volumes using any/other nfs > client and check if you are able to perform write operations. Also > please collect the gluster nfs log while doing so. > > Thanks, > Soumya > > On 01/22/2015 08:18 AM, Peter Auyeung wrote: >> Hi, >> >> We have been having 5 sybase servers doing dump/export to Gluster NFS >> for couple months and yesterday it started to give us these error on not >> able to write files >> >> The gluster NFS export is not full and we can still move and write files >> as sybase unix user from the sybase servers. >> >> There are no error logs on gluster nfs nor the bricks and etc-glusterfs >> logs and no nfs client error on the sybase servers neither. >> >> The NFS export was a replica 2 volume (3x2) >> >> I created another NFS export from same gluster but a distributed only >> volume and still giving out the same error. >> >> Any Clue? >> >> Thanks >> Peter >> >> Jan 20 20:04:17 2015: Backup Server: 6.53.1.1: OPERATOR: Volume on >> device '/dbbackup01/db/full/pr_rssd_id_repsrv_rssd.F01-20-20-04.e' >> cannot be opened for write access. Mount another volume. >> Jan 20 20:04:17 2015: Backup Server: 6.78.1.1: EXECUTE sp_volchanged >> @session_id = 87, >> @devname = >> '/dbbackup01/db/full/pr_rssd_id_repsrv_rssd.F01-20-20-04.e', >> @action = { 'PROCEED' | 'RETRY' | 'ABORT' } >> Jan 20 20:04:26 2015: Backup Server: 6.53.1.1: OPERATOR: Volume on >> device '/dbbackup01/db/full/pr_rssd_id_repsrv_rssd.F01-20-20-04.a' >> cannot be opened for write access. Mount another volume. >> Jan 20 20:04:26 2015: Backup Server: 6.78.1.1: EXECUTE sp_volchanged >> @session_id = 87, >> @devname = >> '/dbbackup01/db/full/pr_rssd_id_repsrv_rssd.F01-20-20-04.a', >> @action = { 'PROCEED' | 'RETRY' | 'ABORT' } >> Jan 20 20:05:41 2015: Backup Server: 6.53.1.1: OPERATOR: Volume on >> device '/dbbackup01/db/full/pr_rssd_id_repsrv_rssd.F01-20-20-04.d' >> cannot be opened for write access. Mount another volume. >> Jan 20 20:05:41 2015: Backup Server: 6.78.1.1: EXECUTE sp_volchanged >> @session_id = 87, >> @devname = >> '/dbbackup01/db/full/pr_rssd_id_repsrv_rssd.F01-20-20-04.d', >> @action = { 'PROCEED' | 'RETRY' | 'ABORT' } >> >> >> >> _______________________________________________ >> Gluster-devel mailing list >> Gluster-devel@xxxxxxxxxxx >> http://www.gluster.org/mailman/listinfo/gluster-devel >> _______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx http://www.gluster.org/mailman/listinfo/gluster-users _______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx http://www.gluster.org/mailman/listinfo/gluster-users