Re: Split brain that is not split brain

[Pranith Kumar Karampuri <pkarampu@xxxxxxxxxx>]

On 09/11/2014 01:13 PM, Ilya Ivanov wrote:

Makes some sense. Yes, I meant "make a backup and delete", rather than just delete.

If I may suggest, putting that debug link somewhere more visible would be be good, too. I wouldn't find without your help.

Justin, where shall we put the doc?

Pranith

Thank you for the assistance.

On Thu, Sep 11, 2014 at 9:14 AM, Pranith Kumar Karampuri <pkarampu@xxxxxxxxxx> wrote:

On 09/11/2014 11:37 AM, Ilya Ivanov wrote:

I don't understand why there's such a complicated process to recover when I can just look at both files, decide which one I need and delete another one.

If the file needs to be deleted the whole file needs to be copied which is fine for small files but for big files like VM images it takes less time if the file already exists and it syncs only the parts of files that are different from the good copy.
One more reason is if the parent directory from which the file is deleted from is the source then self-heal will delete the file from other directory rather than creating it. SO instead of deleting the file may be it is a better practise to make a copy of the file somewhere and delete it. We shall update the document as well with this new information. Thanks for the feedback. In 3.7 it is going to be simplified. We are giving a command to fix the split-brains where the user gets to choose the file and it will do the rest.

Pranith

On Thu, Sep 11, 2014 at 7:56 AM, Pranith Kumar Karampuri <pkarampu@xxxxxxxxxx> wrote:

On 09/11/2014 09:29 AM, Ilya Ivanov wrote:

Right... I deleted it and now all appears to be fine.

Still, could you please elaborate on "gfid split-brain"?

Could you go through https://github.com/gluster/glusterfs/blob/master/doc/debugging/split-brain.md
Let us know if you would like something to be more clearer and we can add that and improve the document.

Pranith

On Thu, Sep 11, 2014 at 5:32 AM, Pranith Kumar Karampuri <pkarampu@xxxxxxxxxx> wrote:

On 09/11/2014 12:16 AM, Ilya Ivanov wrote:

Any insight?

Was the other file's gfid d3def9e1-c6d0-4b7d-a322-b5019305182e?
Could you check if this file exists in <brick>/.glusterfs/d3/de/
When a file is deleted this file also needs to be deleted if there are no more hardlinks to the file

Pranith

On Tue, Sep 9, 2014 at 8:35 AM, Ilya Ivanov <bearwere@xxxxxxxxx> wrote:

What's a gfid split-brain and how is it different from "normal" split-brain?

I accessed the file with stat, but "heal info" still shows "Number of entries: 1"

[root@gluster1 gluster]# getfattr -d -m. -e hex gv01/123
# getfattr -d -m. -e hex gv01/123
# file: gv01/123
trusted.afr.gv01-client-0=0x000000000000000000000000
trusted.afr.gv01-client-1=0x000000000000000000000000
trusted.gfid=0x35f86f4561134ba0bd1b94ef70179d4d

[root@gluster1 gluster]# getfattr -d -m. -e hex gv01
# file: gv01
trusted.afr.gv01-client-0=0x000000000000000000000000
trusted.afr.gv01-client-1=0x000000000000000000000000
trusted.gfid=0x00000000000000000000000000000001
trusted.glusterfs.dht=0x000000010000000000000000ffffffff
trusted.glusterfs.volume-id=0x31a2c4c486ca4344b838d2c2e6c716c1

On Tue, Sep 9, 2014 at 8:19 AM, Pranith Kumar Karampuri <pkarampu@xxxxxxxxxx> wrote:

On 09/09/2014 11:35 AM, Ilya Ivanov wrote:

Ahh, thank you, now I get it. I deleted it on one node and it replicated to another one. Now I get the following output:

[root@gluster1 var]# gluster volume heal gv01 info
Brick gluster1:/home/gluster/gv01/
<gfid:d3def9e1-c6d0-4b7d-a322-b5019305182e>
Number of entries: 1

Brick gluster2:/home/gluster/gv01/
Number of entries: 0

Is it normal? Why the number of entries isn't reset to 0?

If you access the file using ls/stat etc, it will be fixed. But before that could you please post the output of 'getfattr -d -m. -e hex file/path/in/backend/brick' and 'getfattr -d -m. -e hex parent/dir/to/file/path/in/backend/brick'

Pranith

And why wouldn't the file show up in split-brain before, anyway?

Gfid split-brains are not shown in heal-info-split-brain yet.

Pranith

On Tue, Sep 9, 2014 at 7:46 AM, Pranith Kumar Karampuri <pkarampu@xxxxxxxxxx> wrote:

On 09/09/2014 01:54 AM, Ilya Ivanov wrote:

Hello.

I've Gluster 3.5.2 on Centos 6. A primitive replicated volume, as describe here. I tried to simulate split-brain by temporarily disconnecting the nodes and creating a file with the same name and different contents. That worked.

The question is, how do I fix it now? All the tutorials suggest deleting the file from one of the nodes. I can't do that, it reports "Input/output error". The file won't even show up in "gluster volume heal gv00 info split-brain". That shows 0 entries.

The deletion needs to happen on one of the bricks, not from the mount point.

Pranith

I can see the file in "gluster volume heal gv00 info heal-failed", though.

--

Ilya.

_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://supercolony.gluster.org/mailman/listinfo/gluster-users

--

Ilya.

--

Ilya.

--

Ilya.

--

Ilya.

--

Ilya.

--

Ilya.

_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://supercolony.gluster.org/mailman/listinfo/gluster-users