Re: replace brick in distributed-dispersed setup

itlinux_team <itlinux@xxxxxxxxx> · Fri, 8 Jul 2016 15:10:20 +0200



    Hi Ashish,

      
      It was an error on my side, nothing gluster related. The kernel
      version I was running had a bug that prevented the fuse module
      from loading and causing the brick replacement errors. After
      upgrading I can confirm that the process of replacing the brick
      works fine both with the brick to be replaced online or after
      killing the brick process.

      
      I have a question though. In a real (non virtual) server, if I
      setup the server in JBOD mode, and a drive fails, does gluster
      kill the brick pid? 

      
      Regards,

      Iñaki.

      
      On 07/08/2016 07:03 AM, Ashish Pandey wrote:

    
        Hi Iñaki

          
          The steps you are following don't have any issue.

          
          I would like to have more information to debug this
            further.

          
          1 - gluster v info <volname>

          
          2 - gluster v status <volname> before and after
            running replace-brick

          
          3 - Brick logs (for this volume only) from
          /var/log/glusterfs/bricks/

        
        4 - glusterd logs /var/log/glusterfs/  starts with
          "usr-local-etc-glusterfs-glusterd-<volname>"

        
        Although it should not matter,  could you also try to
          replace a brick without killing that brick process?

        
        Ashish

        
        From: "itlinux_team"
          <itlinux@xxxxxxxxx>

          To: gluster-users@xxxxxxxxxxx

          Sent: Wednesday, July 6, 2016 4:33:54 PM

          Subject:  replace brick in
          distributed-dispersed setup

          
          Hi all,

          
          I'm doing some testings with glusterfs in a virtualized
          environment 

          running a 3 x (8 + 4) distributed-dispersed volume simulating
          a 3 node 

          cluster with 12 drives per node configuration. The system
          versions are:

          
          OS: Debian jessie kernel 3.16

          
          Gluster:  3.8.0-2 installed from the gluster.org debian
          repository

          
          I have tested the node failure scenario while some clients are
          running 

          some read/write operations and the setup works as expected.
          Now I'm 

          trying to test how to replace a faulty drive on this setup,
          however I'm 

          not able to replace a brick. To test it I have:

          
          1: Find the pid of the brick I'd like to 'fail' and kill the
          process. 

          (tried removing the drive from the host but that would make
          the whole 

          guest unresponsive)

          
          2: Attach a new virtual drive, format and mount it

          
          3: Try the gluster volume replace-brick command

          
          And I'm getting the following error:

          
          gluster volume replace-brick vol_1
          glusterserver1:/ext/bricks/brick-1 

          glusterserver1:/ext/bricks/brick-13 commit force

          volume replace-brick: failed: Fuse unavailable

            Replace-brick failed

          
          I assume I'm doing something wrong but don't know what
          exactly. Looking 

          in the documentation I have not found information about brick
          

          replacement in distributed-dispersed setups.

          
          Thanks!

          
          Iñaki

          
          _______________________________________________

          Gluster-users mailing list

          Gluster-users@xxxxxxxxxxx

          http://www.gluster.org/mailman/listinfo/gluster-users
        

_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-users