Hi @all, since a few hours we have a problem with duplicate files in one directory (and it’s subdirectories) of our volume. It looks like that the error occurs only on client side (on the server side the folder structure looks fine) and that only files (not directories) are affected. ------------------------------------------------------- $ ls -l /glusterfs/gv0/affected-directory/ total 132 [...] -rw-r--r-- 1 user group 19878 May 16 2017 Error_404.html -rw-r--r-- 1 user group 19878 May 16 2017 Error_404.html […] -rw-rw-r-- 1 user group 91 Mar 10 2017 robots.txt -rw-rw-r-- 1 user group 91 Mar 10 2017 robots.txt […] ------------------------------------------------------- an remount didn’t fix the problem but after replacing the mount directive everything works fine, but I don’t no why (and if there are hidden side effects) and `$ gluster volume heal gv0 info` shows also still an error. So I also started a full heal request via `$ gluster volume heal gv0 full` (hopefully it will fix the whole thing). ------------------------------------------------------- old ------------------------------------------------------- host1:/gv0 /glusterfs/gv0 glusterfs defaults,_netdev,log-level=WARNING 0 0 ------------------------------------------------------- ------------------------------------------------------- new ------------------------------------------------------- /etc/glusterfs/datastore-vg0.vol /glusterfs/gv0 glusterfs defaults,_netdev,log-level=WARNING 0 0 ------------------------------------------------------- Does anyone have an idea why this happened? Thanks in advance. Best regards, Gerrit ------------------------------------------------------- $ cat /etc/glusterfs/datastore-vg0.vol volume remote1 type protocol/client option transport-type tcp option remote-host host1 option remote-subvolume /export/xvda3/brick end-volume volume remote2 type protocol/client option transport-type tcp option remote-host host2 option remote-subvolume /export/xvda3/brick end-volume volume remote3 type protocol/client option transport-type tcp option remote-host host3 option remote-subvolume /export/xvda3/brick end-volume volume remote4 type protocol/client option transport-type tcp option remote-host host4 option remote-subvolume /export/xvda3/brick end-volume volume replicate type cluster/replicate subvolumes remote1 remote2 remote3 remote4 end-volume volume writebehind type performance/write-behind #option window-size 1MB subvolumes replicate end-volume volume cache type performance/io-cache option cache-size 512MB subvolumes writebehind end-volume ------------------------------------------------------- ------------------------------------------------------- Server - glusterfs 3.8.7 ------------------------------------------------------- $ sudo gluster volume info Volume Name: gv0 Type: Distributed-Replicate Volume ID: xxxxxxxxxxxxxxxxxxx Status: Started Snapshot Count: 0 Number of Bricks: 2 x 2 = 4 Transport-type: tcp Bricks: Brick1: host1:/export/xvda3/brick Brick2: host2:/export/xvda3/brick Brick3: host3:/export/xvda3/brick Brick4: host4:/export/xvda3/brick Options Reconfigured: transport.address-family: inet performance.readdir-ahead: on nfs.disable: off performance.cache-size: 512MB performance.client-io-threads: on performance.io-thread-count: 64 performance.read-ahead: off performance.cache-refresh-timeout: 1 ------------------------------------------------------- ------------------------------------------------------- $ gluster volume heal gv0 info Brick host1:/export/xvda3/brick Status: Connected Number of entries: 0 Brick host2:/export/xvda3/brick Status: Connected Number of entries: 0 Brick host3:/export/xvda3/brick Status: Connected Number of entries: 0 Brick host4:/export/xvda3/brick <gfid:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxx> Status: Connected Number of entries: 1 ------------------------------------------------------- ------------------------------------------------------- $ gluster volume heal gv0 statistics [...] Ending time of crawl: Thu Oct 10 17:30:54 2019 Type of crawl: INDEX No. of entries healed: 0 No. of entries in split-brain: 0 No. of heal failed entries: 2 Starting time of crawl: Thu Oct 10 17:40:54 2019 Ending time of crawl: Thu Oct 10 17:40:55 2019 Type of crawl: INDEX No. of entries healed: 0 No. of entries in split-brain: 0 No. of heal failed entries: 1 [...] Starting time of crawl: Thu Oct 10 17:28:01 2019 Crawl is in progress Type of crawl: FULL No. of entries healed: 0 No. of entries in split-brain: 0 No. of heal failed entries: 0 ------------------------------------------------------- ________ Community Meeting Calendar: APAC Schedule - Every 2nd and 4th Tuesday at 11:30 AM IST Bridge: https://bluejeans.com/118564314 NA/EMEA Schedule - Every 1st and 3rd Tuesday at 01:00 PM EDT Bridge: https://bluejeans.com/118564314 Gluster-users mailing list Gluster-users@xxxxxxxxxxx https://lists.gluster.org/mailman/listinfo/gluster-users