Issues in AFR and self healing

Pablo Schandin <pablo.schandin@xxxxxxxxxxx> · Fri, 10 Aug 2018 14:55:54 -0300



    Hello everyone!
    I'm having some trouble with something but I'm not quite sure of
      with what yet. I'm running GlusterFS 3.12.6 on Ubuntu 16.04. I
      have two servers (nodes) in the cluster in a replica mode. Each
      server has 2 bricks. As the servers are KVM running several VMs,
      one brick has some VMs locally defined in it and the second brick
      is the replicated from the other server. It has data but not
      actual writing is being done except for the replication.

    
                                Server 1                           
                                              Server 2

      Volume 1 (gv1): Brick 1 defined VMs (read/write)    ---->   
                    Brick 1 replicated qcow2 files

      Volume 2 (gv2): Brick 2 replicated qcow2 files        <-----   
                   Brick 2 defined VMs (read/write)
    So, the main issue arose when I got a nagios alarm that warned
      about a file listed to be healed. And then it disappeared. I came
      to find out that every 5 minutes, the self heal daemon triggers
      the healing and this fixes it. But looking at the logs I have a
      lot of entries in the glustershd.log file like this:
    [2018-08-09 14:23:37.689403] I [MSGID: 108026]
        [afr-self-heal-common.c:1656:afr_log_selfheal]
        0-gv1-replicate-0: Completed data selfheal on
        407bd97b-e76c-4f81-8f59-7dae11507b0c. sources=[0]  sinks=1 

        [2018-08-09 14:44:37.933143] I [MSGID: 108026]
        [afr-self-heal-common.c:1656:afr_log_selfheal]
        0-gv2-replicate-0: Completed data selfheal on
        73713556-5b63-4f91-b83d-d7d82fee111f. sources=[0]  sinks=1 

    
    The qcow2 files are being healed several times a day (up to 30 in
      occasions). As I understand, this means that a data heal occurred
      on file with gfid 407b... and 7371... in source to sink. Local
      server to replica server? Is it OK for the shd to heal files in
      the replicated brick that supposedly has no writing on it besides
      the mirroring? How does that work?
    How does afr replication work? The file with gfid 7371... is the
      qcow2 root disk of an owncloud server with 17GB of data. It does
      not seem to be that big to be a bottleneck of some sort, I think.
    Also, I was investigating the directory tree in
      brick/.glusterfs/indices and I notices that both in xattrop and
      dirty I always have a file created named xattrop-xxxxxx and
      dirty-xxxxxx. I read that the xattrop file is like a parent file
      or handle to reference other files created there as hardlinks with
      gfid name for the shd to heal. Is the same case as the ones in the
      dirty dir?
    Any help will be greatly appreciated it. Thanks!

    
    Pablo.

    
Attachment:
smime.p7s

Description: S/MIME Cryptographic Signature
_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
https://lists.gluster.org/mailman/listinfo/gluster-users