Re: query about why glustershd can not afr_selfheal_recreate_entry because of "afr: Prevent null gfids in self-heal entry re-creation"

Ravishankar N <ravishankar@xxxxxxxxxx> · Tue, 16 Jan 2018 11:14:16 +0530



    + gluster-devel
    

    On 01/15/2018 01:41 PM, Zhou, Cynthia
      (NSB - CN/Hangzhou) wrote:

    
          Hi glusterfs
            expert,
                  Good
            day,
                  When I
            do some test about glusterfs self-heal I find following
            prints showing when dir/file type get error it cannot get
            self-healed.
          Could
              you help to check if it is an expected behavior ? because
              I find the code change https://review.gluster.org/#/c/17981/
              add check for iatt->ia_type,  so what if
              a file’s ia_type get corrupted ? in this case it should
              not get self-healed ? 
        
    
    Yes, without knowing the ia-type ,
      afr_selfheal_recreate_entry () cannot decide what type of FOP to
      do (mkdir/link/mknod ) to create the appropriate file on the sink.
      You would need to find out why the source brick is not returning
      valid ia_type. i.e. why replies[source].poststat is not valid.

      Thanks,

      Ravi

        
          Thanks!
          //////////////////heal
            info output////////////////////////////
          [root@sn-0:/home/robot]
          # gluster v heal
            export info
          Brick
            sn-0.local:/mnt/bricks/export/brick
          Status:
            Connected
          Number of
            entries: 0
           
          Brick
            sn-1.local:/mnt/bricks/export/brick
          /testdir - Is in
            split-brain
           
          Status:
            Connected
          Number of
            entries: 1
          //////////////////////////////////////////sn-1
            glustershd
            log///////////////////////////////////////////////////
           
          [2018-01-15
            03:53:40.011422] I [MSGID: 108026]
            [afr-self-heal-entry.c:887:afr_selfheal_entry_do]
            0-export-replicate-0: performing entry selfheal on
            b217d6af-4902-4f18-9a69-e0ccf5207572
          [2018-01-15
            03:53:40.013994] W [MSGID: 114031]
            [client-rpc-fops.c:2860:client3_3_lookup_cbk]
            0-export-client-1: remote operation failed. Path: (null)
            (00000000-0000-0000-0000-000000000000) [No data available]
          [2018-01-15
            03:53:40.014025] E [MSGID: 108037]
            [afr-self-heal-entry.c:92:afr_selfheal_recreate_entry]
            0-export-replicate-0: Invalid ia_type (0)
              or gfid(00000000-0000-0000-0000-000000000000). source
              brick=1, pargfid=00000000-0000-0000-0000-000000000000,
              name=IORFILE_82_2
          //////////////////////////////////////gdb
            attached to sn-1
            glustershd/////////////////////////////////////////////
          root@sn-1:/var/log/glusterfs]
          # gdb attach
            2191
          GNU gdb (GDB)
            8.0.1
          Copyright (C)
            2017 Free Software Foundation, Inc.
          License GPLv3+:
            GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
          This is free
            software: you are free to change and redistribute it.
          There is NO
            WARRANTY, to the extent permitted by law.  Type "show
            copying"
          and "show
            warranty" for details.
          This GDB was
            configured as "x86_64-linux-gnu".
          Type "show
            configuration" for configuration details.
          For bug
            reporting instructions, please see:
          <http://www.gnu.org/software/gdb/bugs/>.
          Find the GDB
            manual and other documentation resources online at:
          <http://www.gnu.org/software/gdb/documentation/>.
          For help, type
            "help".
          Type "apropos
            word" to search for commands related to "word"...
          attach: No such
            file or directory.
          Attaching to
            process 2191
          [New LWP 2192]
          [New LWP 2193]
          [New LWP 2194]
          [New LWP 2195]
          [New LWP 2196]
          [New LWP 2197]
          [New LWP 2239]
          [New LWP 2241]
          [New LWP 2243]
          [New LWP 2245]
          [New LWP 2247]
          [Thread
            debugging using libthread_db enabled]
          Using host
            libthread_db library "/lib64/libthread_db.so.1".
          0x00007f90aca037bd
            in __pthread_join (threadid=140259279345408,
            thread_return=0x0) at pthread_join.c:90
          90     
            pthread_join.c: No such file or directory.
          (gdb) break
            afr_selfheal_recreate_entry
          Breakpoint 1 at
            0x7f90a3b56dec: file afr-self-heal-entry.c, line 73.
          (gdb) c
          Continuing.
          [Switching to
            Thread 0x7f90a1b8e700 (LWP 2241)]
           
          Thread 9
            "glustershdheal" hit Breakpoint 1,
            afr_selfheal_recreate_entry (frame=0x7f90980018d0, dst=0,
            source=1, sources=0x7f90a1b8ceb0 "", dir=0x7f9098011940,
            name=0x7f909c015d48 "IORFILE_82_2", 
             
            inode=0x7f9098001bd0, replies=0x7f90a1b8c890) at
            afr-self-heal-entry.c:73
          73     
            afr-self-heal-entry.c: No such file or directory.
          (gdb) n
          74      in
            afr-self-heal-entry.c
          (gdb) n
          75      in
            afr-self-heal-entry.c
          (gdb) n
          76      in
            afr-self-heal-entry.c
          (gdb) n
          77      in
            afr-self-heal-entry.c
          (gdb) n
          78      in
            afr-self-heal-entry.c
          (gdb) n
          79      in
            afr-self-heal-entry.c
          (gdb) n
          80      in
            afr-self-heal-entry.c
          (gdb) n
          81      in
            afr-self-heal-entry.c
          (gdb) n
          82      in
            afr-self-heal-entry.c
          (gdb) n
          83      in
            afr-self-heal-entry.c
          (gdb) n
          85      in
            afr-self-heal-entry.c
          (gdb) n
          86      in
            afr-self-heal-entry.c
          (gdb) n
          87      in
            afr-self-heal-entry.c
          (gdb) print
            iatt->ia_type
          $1 = IA_INVAL
          (gdb) print
            gf_uuid_is_null(iatt->ia_gfid)
          $2 = 1
          (gdb) bt
          #0 
            afr_selfheal_recreate_entry (frame=0x7f90980018d0, dst=0,
            source=1, sources=0x7f90a1b8ceb0 "", dir=0x7f9098011940,
            name=0x7f909c015d48 "IORFILE_82_2", inode=0x7f9098001bd0,
            replies=0x7f90a1b8c890)
              at
            afr-self-heal-entry.c:87
          #1 
            0x00007f90a3b57d20 in __afr_selfheal_merge_dirent
            (frame=0x7f90980018d0, this=0x7f90a4024610,
            fd=0x7f9098413090, name=0x7f909c015d48 "IORFILE_82_2",
            inode=0x7f9098001bd0, 
             
            sources=0x7f90a1b8ceb0 "", healed_sinks=0x7f90a1b8ce70
            "\001\001A\230\220\177", locked_on=0x7f90a1b8ce50
            "\001\001\270\241\220\177", replies=0x7f90a1b8c890) at
            afr-self-heal-entry.c:360
          #2 
            0x00007f90a3b57da5 in __afr_selfheal_entry_dirent
            (frame=0x7f90980018d0, this=0x7f90a4024610,
            fd=0x7f9098413090, name=0x7f909c015d48 "IORFILE_82_2",
            inode=0x7f9098001bd0, source=-1, 
             
            sources=0x7f90a1b8ceb0 "", healed_sinks=0x7f90a1b8ce70
            "\001\001A\230\220\177", locked_on=0x7f90a1b8ce50
            "\001\001\270\241\220\177", replies=0x7f90a1b8c890) at
            afr-self-heal-entry.c:379
          #3 
            0x00007f90a3b5881f in afr_selfheal_entry_dirent
            (frame=0x7f90980018d0, this=0x7f90a4024610,
            fd=0x7f9098413090, name=0x7f909c015d48 "IORFILE_82_2",
            parent_idx_inode=0x0, subvol=0x7f90a4022240, 
             
            full_crawl=_gf_true) at afr-self-heal-entry.c:610
          #4 
            0x00007f90a3b58da8 in afr_selfheal_entry_do_subvol
            (frame=0x7f90980110f0, this=0x7f90a4024610,
            fd=0x7f9098413090, child=1) at afr-self-heal-entry.c:742
          #5 
            0x00007f90a3b5953a in afr_selfheal_entry_do
            (frame=0x7f90980110f0, this=0x7f90a4024610,
            fd=0x7f9098413090, source=-1, sources=0x7f90a1b8d810 "", 
             
            healed_sinks=0x7f90a1b8d7d0 "\001\001\270\241\220\177") at
            afr-self-heal-entry.c:908
          #6 
            0x00007f90a3b59b79 in __afr_selfheal_entry
            (frame=0x7f90980110f0, this=0x7f90a4024610,
            fd=0x7f9098413090, locked_on=0x7f90a1b8d920 "\001\001\\Z")
            at afr-self-heal-entry.c:1002
          #7 
            0x00007f90a3b5a051 in afr_selfheal_entry
            (frame=0x7f90980110f0, this=0x7f90a4024610,
            inode=0x7f9098011940) at afr-self-heal-entry.c:1112
          #8 
            0x00007f90a3b519b1 in afr_selfheal_do (frame=0x7f90980110f0,
            this=0x7f90a4024610, gfid=0x7f90a1b8db20 "\262\027֯I\002O\030\232i\340\314\365 urP۸\241\220\177")
            at afr-self-heal-common.c:2459
          #9 
            0x00007f90a3b51aa7 in afr_selfheal (this=0x7f90a4024610,
            gfid=0x7f90a1b8db20 "\262\027֯I\002O\030\232i\340\314\365
            urP۸\241\220\177") at
            afr-self-heal-common.c:2500
          #10
            0x00007f90a3b5cf1f in afr_shd_selfheal
            (healer=0x7f90a4033510, child=1, gfid=0x7f90a1b8db20
            "\262\027֯I\002O\030\232i\340\314\365
            urP۸\241\220\177") at
            afr-self-heald.c:334
          #11
            0x00007f90a3b5d2c8 in afr_shd_index_heal
            (subvol=0x7f90a4022240, entry=0x7f909c0169c0,
            parent=0x7f90a1b8dde0, data="" at
            afr-self-heald.c:431
          #12
            0x00007f90adc74654 in syncop_mt_dir_scan
            (frame=0x7f90a407b4f0, subvol=0x7f90a4022240,
            loc=0x7f90a1b8dde0, pid=-6, data=""
            fn=0x7f90a3b5d17c <afr_shd_index_heal>, 
             
            xdata=0x7f90a4002380, max_jobs=1, max_qlen=1024) at
            syncop-utils.c:407
          #13
            0x00007f90a3b5d4e1 in afr_shd_index_sweep
            (healer=0x7f90a4033510, vgfid=0x7f90a3b84f38
            "glusterfs.xattrop_index_gfid") at afr-self-heald.c:481
          #14
            0x00007f90a3b5d5bd in afr_shd_index_sweep_all
            (healer=0x7f90a4033510) at afr-self-heald.c:504
          #15
            0x00007f90a3b5d894 in afr_shd_index_healer
            (data="" at afr-self-heald.c:584
          #16
            0x00007f90aca024a5 in start_thread (arg=0x7f90a1b8e700) at
            pthread_create.c:465
          #17
            0x00007f90ac2e959f in clone () at
            ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
          (gdb)
           
           
          Best regards,

              Cynthia （周琳）
          MBB SM HETRAN SW3 MATRIX  
          Storage         

              Mobile: +86 (0)18657188311
           
           
_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
http://lists.gluster.org/mailman/listinfo/gluster-devel