Re: Bareos backup from Gluster mount

Pranith Kumar Karampuri <pkarampu@xxxxxxxxxx> · Thu, 30 Jul 2015 13:05:50 +0530



    The errors you gave in the log seem to be coming from Filesystem. So
    I am pretty sure something is coming from either vfs or gluster
    itself. How is the backup job run? what are the uid/gid of the
    process? Could you do an strace -ff -f -p <pid-of-backup-job>
    -o /path/to/log so that we can find out more about what exactly is
    happening. It will create files of the format
    /path/to/log.<pid> This should help us figure out more about
    what could be happening.

    
    Pranith

    
    On 07/29/2015 11:31 PM, Ryan Clough
      wrote:

    
        Thank you Pranith but
          nothing interesting is logged to any of the Gluster logs.
          Perhaps this is an issue with Bareos. Just though I would ping
          this wonderful group for any wisdom. Thanks for taking the
          time to help. If I discover the issue I will post back here.

        
                ___________________________________________

                      ¯\_(ツ)_/¯

                      Ryan Clough

                      Information Systems

                      Decision Sciences International
                        Corporation
              
            
        On Wed, Jul 29, 2015 at 5:53 AM,
          Pranith Kumar Karampuri <pkarampu@xxxxxxxxxx> wrote:

          
             hi Ryan,

                     What do you see in the logs of glusterfs mount and
              bricks? Do you think it is possible for you to attach
              those logs to this thread so that we can see what could be
              going on?

              
              Pranith
              
                
                  On 07/28/2015 02:32 AM, Ryan Clough wrote:

                  
                        Hello,

                          
                        I have
                          cross-posted this question in the bareos-users
                          mailing list.

                        
                        Wondering
                          if anyone has tried this because I am unable
                          to backup data that is mounted via Gluster
                          Fuse or Gluster NFS. Basically, I have the
                          Gluster volume mounted on the Bareos Director
                          which also has the tape changer attached.

                          
                        Here
                          is some information about versions:

                        
                        Bareos
                          version 14.2.2

                        
                        Gluster
                          version 3.7.2

                        
                        Scientific
                          Linux version 6.6

                          
                        Our
                          Gluster volume consists of two nodes in
                          distribute only. Here is the configuration of
                          our volume:

                          [root@hgluster02 ~]# gluster volume info

                           
                          Volume Name: export_volume

                          Type: Distribute

                          Volume ID:
                          c74cc970-31e2-4924-a244-4c70d958dadb

                          Status: Started

                          Number of Bricks: 2

                          Transport-type: tcp

                          Bricks:

                          Brick1: hgluster01:/gluster_data

                          Brick2: hgluster02:/gluster_data

                          Options Reconfigured:

                          performance.io-thread-count: 24

                          server.event-threads: 20

                          client.event-threads: 4

                          performance.readdir-ahead: on

                          features.inode-quota: on

                          features.quota: on

                          nfs.disable: off

                          auth.allow:
                          192.168.10.*,10.0.10.*,10.8.0.*,10.2.0.*,10.0.60.*

                          server.allow-insecure: on

                          server.root-squash: on

                          performance.read-ahead: on

                          features.quota-deem-statfs: on

                          diagnostics.brick-log-level: WARNING

                          
                        When I
                          try to backup a directory from Gluster Fuse or
                          Gluster NFS mount and I monitor the network
                          communication I only see data being pulled
                          from the hgluster01 brick. When the job
                          finishes Bareos thinks that it completed
                          without error but included in the messages for
                          the job are lots and lots of permission denied
                          errors like this:

                          15-Jul 02:03 ripper.red.dsic.com-fd JobId
                          613:      Cannot open
                          "/export/rclough/psdv-2014-archives-2/scan_111.tar.bak":
                          ERR=Permission denied.

                          15-Jul 02:03 ripper.red.dsic.com-fd JobId
                          613:      Cannot open
                          "/export/rclough/psdv-2014-archives-2/run_219.tar.bak":
                          ERR=Permission denied.

                          15-Jul 02:03 ripper.red.dsic.com-fd JobId
                          613:      Cannot open
                          "/export/rclough/psdv-2014-archives-2/scan_112.tar.bak":
                          ERR=Permission denied.

                          15-Jul 02:03 ripper.red.dsic.com-fd JobId
                          613:      Cannot open
                          "/export/rclough/psdv-2014-archives-2/run_220.tar.bak":
                          ERR=Permission denied.

                          15-Jul 02:03 ripper.red.dsic.com-fd JobId
                          613:      Cannot open
                          "/export/rclough/psdv-2014-archives-2/scan_114.tar.bak":
                          ERR=Permission denied.

                          
                        At
                          first I thought this might be a root-squash
                          problem but, if I try to read/copy a file
                          using the root user from the Bareos server
                          that is trying to do the backup, I can read
                          files just fine.

                          
                        When
                          the job finishes is reports that it finished
                          "OK -- with warnings" but, again the log for
                          the job is filled with "ERR=Permission denied"
                          messages. In my opinion, this job did not
                          finish OK and should be Failed. Some of the
                          files from the HGluster02 brick are backed up
                          but all of the ones with permission errors do
                          not. When I restore the job, all of the files
                          with permission errors are empty.

                          
                        Has
                          anyone successfully used Bareos to backup data
                          from Gluster mounts? This is an important use
                          case for us because this is the largest single
                          volume that we have to prepare large amounts
                          of data to be archived.

                        
                        Thank you for your time,
                      
                        
                              ___________________________________________

                                    ¯\_(ツ)_/¯

                                    Ryan Clough

                                    Information Systems

                                    Decision Sciences
                                      International Corporation
                            
                          
                This email and its contents
                    are confidential. If you are not the intended
                    recipient, please do not disclose or use the
                    information within this email or its attachments. If
                    you have received this email in error, please report
                    the error to the sender by return email and delete
                    this communication from your records.
                

                _______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-users
              
              
      This email and its contents are
          confidential. If you are not the intended recipient, please do
          not disclose or use the information within this email or its
          attachments. If you have received this email in error, please
          report the error to the sender by return email and delete this
          communication from your records.
    
    
_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-users