There is a similar error reported in https://access.redhat.com/solutions/5518661 which could indicate a possible problem in a memory area named ‘lru’ .
Have you noticed any ECC errors in dmesg/IPMI of the system ?
At least I would reboot the node and run hardware diagnostics to check that everything is fine.
Best Regards,
Best Regards,
Strahil Nikolov
On Tuesday, July 25, 2023, 4:31 AM, W Kern <wkmail@xxxxxxxxx> wrote:
we have an older 2+1 arbiter gluster cluster running 6.10 on Ubuntu18LTSIt has run beautifully for years. Only occaisionally needing attentionas drives have died, etcEach peer has two volumes. G1 and G2 with a shared 'gluster' network.Since July 1st one of the peers for one volume is spewing the logfile/var-lib-G1.log with the following errors.The volume (G2) is not showing this nor are there issue with other peerand the arbiter for the G1 volume.So its one machine with one volume that has the problem. There havebeen NO issues with the volumes themselves.It simply a matter of the the logfiles generating GBs of entries everyhour (which is how we noticed it when we started running out of log space).According to google there are mentions of this error, but that it wasfixed in the 6.x series. I can find no other mentions.I have tried restarting glusterd with no change. there doesn't seem tobe any hardware issues.I am wondering if perhaps this is an XFS file corruption issue and if Iwere to unmount the Gluster run xfs_repair and bring it back, that wouldsolve the issue.Any other suggestions?[2023-07-21 18:51:38.260507] W [inode.c:1638:inode_table_prune](-->/usr/lib/x86_64-linux-gnu/glusterfs/6.10/xlator/features/shard.so(+0x21b47)[0x7fb261c13b47]-->/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(inode_unref+0x36)[0x7fb26947f416]-->/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(+0x3337a)[0x7fb26947f37a] ) 0-GLB1image-shard: Empty inode lru list found butwith (-2) lru_size[2023-07-21 18:51:38.261231] W [inode.c:1638:inode_table_prune](-->/usr/lib/x86_64-linux-gnu/glusterfs/6.10/xlator/mount/fuse.so(+0xba51)[0x7fb266cdca51]-->/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(inode_unref+0x36)[0x7fb26947f416]-->/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(+0x3337a)[0x7fb26947f37a] ) 0-fuse: Empty inode lru list found but with (-2) lru_size[2023-07-21 18:51:38.261377] W [inode.c:1638:inode_table_prune](-->/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(loc_wipe+0x12)[0x7fb26946bd72]-->/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(inode_unref+0x36)[0x7fb26947f416]-->/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(+0x3337a)[0x7fb26947f37a] ) 0-GLB1image-shard: Empty inode lru list found butwith (-2) lru_size[2023-07-21 18:51:38.261806] W [inode.c:1638:inode_table_prune](-->/usr/lib/x86_64-linux-gnu/glusterfs/6.10/xlator/cluster/replicate.so(+0x5ca57)[0x7fb26213ba57]-->/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(inode_unref+0x36)[0x7fb26947f416]-->/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(+0x3337a)[0x7fb26947f37a] ) 0-GLB1image-replicate-0: Empty inode lru list foundbut with (-2) lru_size[2023-07-21 18:51:38.261933] W [inode.c:1638:inode_table_prune](-->/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(fd_unref+0x1ef)[0x7fb269495eaf]-->/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(inode_unref+0x36)[0x7fb26947f416]-->/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(+0x3337a)[0x7fb26947f37a] ) 0-GLB1image-client-1: Empty inode lru list found butwith (-2) lru_size[2023-07-21 18:51:38.262684] W [inode.c:1638:inode_table_prune](-->/usr/lib/x86_64-linux-gnu/glusterfs/6.10/xlator/cluster/replicate.so(+0x5ca57)[0x7fb26213ba57]-->/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(inode_unref+0x36)[0x7fb26947f416]-->/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(+0x3337a)[0x7fb26947f37a] ) 0-GLB1image-replicate-0: Empty inode lru list foundbut with (-2) lru_size-wk________Community Meeting Calendar:Schedule -Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTCGluster-users mailing list
________ Community Meeting Calendar: Schedule - Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC Bridge: https://meet.google.com/cpu-eiue-hvk Gluster-users mailing list Gluster-users@xxxxxxxxxxx https://lists.gluster.org/mailman/listinfo/gluster-users
