I updated from 3.7.6 to 3.7.8 a few days ago, and now it looks like a number of things are broken including healing.
This is a cluster of 3 servers. One server is Ubuntu 14.04 using the PPA repo, and the other two are Proxmox 4 using the Debian Jessie repo.
"heal info" and "heal statistics" do not show any healing activity; everything shows as zero. But I have broken files that are not getting healed.
Doing "heal", "heal full", and "heal enable" all say success. But none seem to fix anything.
I don't know what to look for in the logs, but I do see a lot of messages in glustershd.log like this:
[2016-02-29 23:13:27.001474] W [MSGID: 108034] [afr-self-heald.c:445:afr_shd_index_sweep] 0-vmdisk2-replicate-0: unable to get index-dir on vmdisk2-client-1
[2016-02-29 23:13:27.001524] W [MSGID: 108034] [afr-self-heald.c:445:afr_shd_index_sweep] 0-public-replicate-0: unable to get index-dir on public-client-3
[2016-02-29 23:13:27.001547] W [MSGID: 108034] [afr-self-heald.c:445:afr_shd_index_sweep] 0-users-replicate-0: unable to get index-dir on users-client-6
[2016-02-29 23:13:27.001876] W [MSGID: 108034] [afr-self-heald.c:445:afr_shd_index_sweep] 0-vmdisk1-replicate-0: unable to get index-dir on vmdisk1-client-2
[2016-02-29 23:13:35.001555] W [MSGID: 108034] [afr-self-heald.c:445:afr_shd_index_sweep] 0-backups-local-replicate-0: unable to get index-dir on backups-local-client-2
On at least one replicated/distributed volume, I see duplicate directory entries (one with the actual file, and one zero-length placeholder)
-rw-rwSrw- 1 root 1004 255744366 Oct 18 2013 S03E05 - The One with Frank Jr.mp4
---------T 1 root 1004 0 Feb 22 08:55 S03E05 - The One with Frank Jr.mp4
-rw-rwSrw- 1 root 1004 255705796 Oct 18 2013 S03E06 - The One with the Flashback.mp4
---------T 1 root 1004 0 Feb 22 08:55 S03E06 - The One with the Flashback.mp4
This is *through the FUSE mount*, not looking directly at the bricks.
Anyone have any ideas on what I should look at? Thanks
- Alan
This is a cluster of 3 servers. One server is Ubuntu 14.04 using the PPA repo, and the other two are Proxmox 4 using the Debian Jessie repo.
"heal info" and "heal statistics" do not show any healing activity; everything shows as zero. But I have broken files that are not getting healed.
Doing "heal", "heal full", and "heal enable" all say success. But none seem to fix anything.
I have tried with entry-self-heal/metdata-self-heal/data-self-heal set both on and off; neither seems to make a difference.
I replaced a brick on a replicated volume. Some of the files are just not being replaced/updated on the second brick. Others have a few blocks written on the second brick but are not complete.
I don't know what to look for in the logs, but I do see a lot of messages in glustershd.log like this:
[2016-02-29 23:13:27.001474] W [MSGID: 108034] [afr-self-heald.c:445:afr_shd_index_sweep] 0-vmdisk2-replicate-0: unable to get index-dir on vmdisk2-client-1
[2016-02-29 23:13:27.001524] W [MSGID: 108034] [afr-self-heald.c:445:afr_shd_index_sweep] 0-public-replicate-0: unable to get index-dir on public-client-3
[2016-02-29 23:13:27.001547] W [MSGID: 108034] [afr-self-heald.c:445:afr_shd_index_sweep] 0-users-replicate-0: unable to get index-dir on users-client-6
[2016-02-29 23:13:27.001876] W [MSGID: 108034] [afr-self-heald.c:445:afr_shd_index_sweep] 0-vmdisk1-replicate-0: unable to get index-dir on vmdisk1-client-2
[2016-02-29 23:13:35.001555] W [MSGID: 108034] [afr-self-heald.c:445:afr_shd_index_sweep] 0-backups-local-replicate-0: unable to get index-dir on backups-local-client-2
On at least one replicated/distributed volume, I see duplicate directory entries (one with the actual file, and one zero-length placeholder)
-rw-rwSrw- 1 root 1004 255744366 Oct 18 2013 S03E05 - The One with Frank Jr.mp4
---------T 1 root 1004 0 Feb 22 08:55 S03E05 - The One with Frank Jr.mp4
-rw-rwSrw- 1 root 1004 255705796 Oct 18 2013 S03E06 - The One with the Flashback.mp4
---------T 1 root 1004 0 Feb 22 08:55 S03E06 - The One with the Flashback.mp4
This is *through the FUSE mount*, not looking directly at the bricks.
Anyone have any ideas on what I should look at? Thanks
- Alan
_______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx http://www.gluster.org/mailman/listinfo/gluster-users
