Re: RHEL6.6 dm-cache and dm-era

Heinz Mauelshagen <heinzm@xxxxxxxxxx> · Fri, 26 Sep 2014 12:33:58 +0200



    Romu,

    
    did you initialize the metadata device before trying to create the
    era target?

    Zeroes in the first KBs will do.

    
    Yes, check for NULL in era_destroy is needed to  handle the error
    path you hit properly.

    
    Heinz

    
    On 09/26/2014 05:16 AM, Romu Hu wrote:

    
      On 2014/9/25 22:57, Heinz Mauelshagen wrote:

      
        Romu,

        
        "dmsetup create MyEra --table "0 20971520 /dev/mapper/mpathbp1
        /dev/mapper/mpathap1 8"

        
        should read

        
        dmsetup create MyEra --table "0 20971520 era
        /dev/mapper/mpathbp1 /dev/mapper/mpathap1 8"

      
      Sorry my bad.

      
      I ran 'dmsetup create MyEra --table "0 20971520 era
      /dev/mapper/mpathbp1 /dev/mapper/mpathap1 8"' but kernel
      (2.6.32-494.el6.i686) oopsed:

      
      Sep 25 18:34:11 localhost kernel: device-mapper: era: sb_check
        failed: magic 0: wanted 2126579579

      Sep 25 18:34:11 localhost kernel: device-mapper: block
        manager: superblock validator check failed for block 0

      Sep 25 18:34:11 localhost kernel: device-mapper: era:
        couldn't read_lock superblock

      Sep 25 18:34:11 localhost kernel: BUG: unable to handle
        kernel NULL pointer dereference at 00000008

      Sep 25 18:34:11 localhost kernel: IP: [<f7e2b3c7>]
        era_destroy+0x7/0x60 [dm_era]Sep 25 18:34:11 localhost kernel:
        *pdpt = 0000000032457001 *pde = 00000003fb5fa067

      Sep 25 18:34:11 localhost kernel: Oops: 0000 [#1] SMP

      Sep 25 18:34:11 localhost kernel: last sysfs file:
        /sys/module/dm_persistent_data/initstate

      
      Perhaps this should be cherry-picked?

      https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=989f26f5ad308f40a95f280bf9cd75e558d4f18d

      
      The log says sb_check failed, anything wrong with the dmsetup
      command parameters?

      
      Thanks

      Romu

      
        On 09/25/2014 04:02 PM, Romu wrote:

        
              2014-09-25 18:19 GMT+08:00 Heinz
                Mauelshagen <heinzm@xxxxxxxxxx>:

                
                      On 09/25/2014 08:10 AM, Romu Hu wrote:

                       On 2014/9/24 12:10,
                        Brassow Jonathan wrote:

                        On Sep 22, 2014, at 7:35
                          AM, Romu wrote:
                          

                            I
                                tried the following command to set up my
                                era target but the command immediately
                                panics the system and the system
                                reboots.
                                

                                # dmsetup create MyEra --table "0
                                  41941903 era
                                  /dev/mapper/VG-CacheDataLV_cmeta
                                  /dev/mapper/VG-OriginLV 4096"

                                
                                The metadata dev and the origin dev
                                  are all part of a LVM cache LV. 
                                  VG-CacheDataLV_cmeta is the cache
                                  metadata LV on the smaller and faster
                                  device, VG-OriginLV is the origin LV
                                  on the faster and slower device,
                                  41941903 is the total sector number of
                                  the device of OriginLV (the LV takes
                                  100% space of the device), 4096 is the
                                  block size of OriginLV, I have run
                                  'mkfs.ext4 /dev/mapper/VG-OriginLV'
                                  before running the dmsetup command.

                                
                                Below is the message in
                                  /var/log/messages after running the
                                  dmsetup comnmand:
                                

                                kernel: device-mapper: era:
                                  sb_check failed: magic 1623043: wanted
                                  2126579579

                                
                                  kernel: device-mapper: block
                                    manager: superblock validator check
                                    failed for block 0
                                  kernel: device-mapper: era:
                                    couldn't read_lock superblock
                                
                                
                                Any idea?
                              

                          Sorry, I haven't used dm-era yet. 
                            However, it does appear that you are perhaps
                            specifying the wrong devices when creating
                            the era device?  Looks like you might be
                            allowing the era target and the cache target
                            to use the same metadata area at the same
                            time - causing them to corrupt each other's
                            metadata area?
                          

                           brassow
                        
                        
                        I think the dmsetup command to set up an era
                        target should be something like this:

                        
                        #
                            dmsetup create MyEra --table "0
                            sector_number era metadeta_dev origin_dev
                            block_size"

                        
                        My questions:

                        
                        1) How to calculate sector_number?  According to
                        https://www.kernel.org/doc/Documentation/device-mapper/era.txt,
                        I guess it should be (4 * nr_blocks) bytes +
                        buffers, but what is nr_blocks and what is
                        buffers?

                      
                     No calculation: it's the size of your era
                    target in sectors. Typically "blockdev --getsz
                    origin_dev"

                      
                       2) Any documentation for
                        dmsetup tables?

                      
                     Look for examples in the kernel source trees
                    Documentaion/device.mapper.

                    
                    table line syntax is: "start_sector
                    length_in_sectors <target>
                    <target_arguments{0,N}>"

                      
                       3) Both my metadata_dev
                        and origin_dev are 10G partitions, is this all
                        right?

                      
                     Metadata device size is plenty but that
                    depends on how many eras with how many updates you
                    want to have.

                      
                       4) My origin_dev is a
                        ext4 filesystem with a block size of 4096, so
                        the block_size in the command line should also
                        be 4096?

                      
                     You may use 4096 = _8_ sectors, you used
                    4096 sectors = 2MB below.

                    It's the granularity the era target housekeeps
                    blocks for.

                      
                        I tried the following command:

                        
                        # dmsetup create MyEra --table "0 160000 era
                          /dev/mapper/mpathap1 /dev/mapper/mpathbp1
                          4096"

                      
                     May not be  same physical device for data
                    and metadata!

                    If it is, thta'd explain your oops.

                    
                    # dmsetup create MyEra --table "0 $(blockdev
                      --getsz /dev/mapper/mpathbp)
                      era whatever_disctinct_metadata_device
                      /dev/mapper/mpathbp1 8"

                    
                    Would use 8 sector block size (which is tiny!) with
                    disctinct metadata and data devices (presumabyl your
                    ext4 sits on /dev/mapper/mpathbp1)

                        
                        Heinz

                      
                Heinz, thank you for your help!
                

                My origin dev is 10G so total sector number
                  is 20971520.  /dev/mapper/mpathbp1 (/dev/dm-6) is the
                  metadata dev, /dev/mapper/mpathap1 (/dev/dm-7) is the
                  origin dev, they are on different LUNs.
                

                I tried the following commands:
                

                # dmsetup create MyEra --table "0 20971520
                  /dev/mapper/mpathbp1 /dev/mapper/mpathap1 8"

                
                # Target type name /dev/mapper/mpathbp1 is too
                  long.
                
                  # Command failed
                
                
                # dmsetup create MyEra --table "0 20971520
                  /dev/dm-6 /dev/dm-7 8"

                
                # device-mapper: reload ioctl on MyEra failed:
                  Invalid argument
                
                  # Command failed
                
                
                And when executing the second command I see the
                  following in /var/log/messages:
                

                  Sep 25 06:37:48 localhost kernel: device-mapper:
                    table: 253:8: /dev/dm-6: unknown target type
                  Sep 25 06:37:48 localhost kernel: device-mapper:
                    ioctl: error adding target to table
                  Sep 25 06:37:48 localhost multipathd: dm-8:
                    remove map (uevent)
                  Sep 25 06:37:48 localhost multipathd: dm-8:
                    devmap not registered, can't remove
                  Sep 25 06:37:48 localhost multipathd: dm-8:
                    remove map (uevent)
                  Sep 25 06:37:48 localhost multipathd: dm-8:
                    devmap not registered, can't remove
                
                
                Then I run 'mkfs.ext4 /dev/dm-6' and tried the
                  second command again but got the same result.
                

                Any idea?
                

                Thanks
                Romu
              
            
          --
dm-devel mailing list
dm-devel@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/dm-devel
        
        
        --
dm-devel mailing list
dm-devel@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/dm-devel
      
      
      --
dm-devel mailing list
dm-devel@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/dm-devel
    
    
    -- 
===============================================================
Heinz Mauelshagen                               +49 2626 141200
Consulting Development Engineer             FAX +49 2626 924446
Red Hat GmbH
Am Sonnenhang 11
56242 Marienrachdorf
Germany                                       heinzm@xxxxxxxxxx
===============================================================
  

--
dm-devel mailing list
dm-devel@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/dm-devel