Hi folks, I recently booted my home LIO FC target box (err, yes, home) with a 3.9.1 kernel, and found some very strange and erroneous behaviour from the target code. The target appears to accept writes but the data never makes it to disk. When the same blocks are read again, the data is the same as it was before the write. This strange behaviour leads me to be able to boot a (FC initator) machine correctly off the target, install a few updates, reboot - just to find the machine snapped back to the state it was in before I booted it up. I did this three times, and began to question my sanity! Creating an LVM logical volume on the initiator machine fails: parker bootc # lvcreate -L 50g -n test vg_parker Failed to activate new LV. Unable to deactivate failed new LV. Manual intervention required. This appears to be because LVM writes metadata to the physical volume, then reads it back to check everything is sane - and finds that it is not. If I make lots of filesystem changes (such as dd if=/dev/zero of=/var/tmp/zero.bin bs=1M) then drop caches with 'echo 3 > /proc/sys/vm/drop_caches', I almost immediately get a journal abort on the filesystem. Note that if I reboot the machine it's snapped back to its previous state so I don't actually get any filesystem corruption at all - just writes that get written to /dev/null. I'm currently remote from the target machine so I can't get on with major work like a git bisect, but I wanted to post this to the ML as soon as possible to get people looking at it. The backstores are all configured nearly identically: #### Parameters for TCM subsystem plugin storage object reference tcm_node --establishdev iblock_3/parker /dev/vg_data/fc_parker tcm_node --setunitserialwithmd iblock_3/parker e8978f4b-e3be-4a4c-bae8-1ec81dbff128 #### ALUA Target Port Groups tcm_node --addaluatpgwithmd iblock_3/parker default_tg_pt_gp 0 echo 3 > /sys/kernel/config/target/core/iblock_3/parker/alua/default_tg_pt_gp/alua_access_type echo 0 > /sys/kernel/config/target/core/iblock_3/parker/alua/default_tg_pt_gp/preferred echo 100 > /sys/kernel/config/target/core/iblock_3/parker/alua/default_tg_pt_gp/nonop_delay_msecs echo 0 > /sys/kernel/config/target/core/iblock_3/parker/alua/default_tg_pt_gp/trans_delay_msecs #### Attributes for /sys/kernel/config/target/core/iblock_3/parker echo 65535 > /sys/kernel/config/target/core/iblock_3/parker/attrib/max_write_same_len echo 0 > /sys/kernel/config/target/core/iblock_3/parker/attrib/unmap_granularity_alignment echo 0 > /sys/kernel/config/target/core/iblock_3/parker/attrib/unmap_granularity echo 0 > /sys/kernel/config/target/core/iblock_3/parker/attrib/max_unmap_block_desc_count echo 0 > /sys/kernel/config/target/core/iblock_3/parker/attrib/max_unmap_lba_count echo 128 > /sys/kernel/config/target/core/iblock_3/parker/attrib/queue_depth echo 8192 > /sys/kernel/config/target/core/iblock_3/parker/attrib/optimal_sectors echo 8192 > /sys/kernel/config/target/core/iblock_3/parker/attrib/fabric_max_sectors echo 512 > /sys/kernel/config/target/core/iblock_3/parker/attrib/block_size echo 0 > /sys/kernel/config/target/core/iblock_3/parker/attrib/emulate_rest_reord echo 0 > /sys/kernel/config/target/core/iblock_3/parker/attrib/is_nonrot echo 1 > /sys/kernel/config/target/core/iblock_3/parker/attrib/enforce_pr_isids echo 0 > /sys/kernel/config/target/core/iblock_3/parker/attrib/emulate_tpws echo 0 > /sys/kernel/config/target/core/iblock_3/parker/attrib/emulate_tpu echo 1 > /sys/kernel/config/target/core/iblock_3/parker/attrib/emulate_tas echo 0 > /sys/kernel/config/target/core/iblock_3/parker/attrib/emulate_ua_intlck_ctrl echo 0 > /sys/kernel/config/target/core/iblock_3/parker/attrib/emulate_write_cache echo 0 > /sys/kernel/config/target/core/iblock_3/parker/attrib/emulate_fua_read echo 1 > /sys/kernel/config/target/core/iblock_3/parker/attrib/emulate_fua_write echo 0 > /sys/kernel/config/target/core/iblock_3/parker/attrib/emulate_dpo echo 0 > /sys/kernel/config/target/core/iblock_3/parker/attrib/emulate_model_alias And the FC targets themselves too: #### qla2xxx Target Ports mkdir -p /sys/kernel/config/target/qla2xxx/21:02:00:e0:8b:d1:67:2c/tpgt_1/lun/lun_0 ln -s /sys/kernel/config/target/qla2xxx/21:02:00:e0:8b:d1:67:2c/tpgt_1/lun/lun_0/../../../../../../target/core/iblock_3/parker /sys/kernel/config/target/qla2xxx/21:02:00:e0:8b:d1:67:2c/tpgt_1/lun/lun_0/63e6a0dab1 #### Attributes for qla2xxx Target Portal Group echo 0 > /sys/kernel/config/target/qla2xxx/21:02:00:e0:8b:d1:67:2c/tpgt_1/attrib/prod_mode_write_protect echo 1 > /sys/kernel/config/target/qla2xxx/21:02:00:e0:8b:d1:67:2c/tpgt_1/attrib/demo_mode_write_protect echo 1 > /sys/kernel/config/target/qla2xxx/21:02:00:e0:8b:d1:67:2c/tpgt_1/attrib/cache_dynamic_acls echo 0 > /sys/kernel/config/target/qla2xxx/21:02:00:e0:8b:d1:67:2c/tpgt_1/attrib/generate_node_acls #### Parameters for qla2xxx Target Portal Group #### qla2xxx Initiator ACLs for qla2xxx Target Portal Group mkdir -p /sys/kernel/config/target/qla2xxx/21:02:00:e0:8b:d1:67:2c/tpgt_1/acls/10:00:00:06:2b:11:3f:a0 #### qla2xxx Initiator ACL authentication information #### qla2xxx Initiator ACL TPG attributes #### qla2xxx Initiator LUN ACLs for iSCSI Target Portal Group mkdir -p /sys/kernel/config/target/qla2xxx/21:02:00:e0:8b:d1:67:2c/tpgt_1/acls/10:00:00:06:2b:11:3f:a0/lun_0 ln -s /sys/kernel/config/target/qla2xxx/21:02:00:e0:8b:d1:67:2c/tpgt_1/acls/10:00:00:06:2b:11:3f:a0/lun_0/../../../../../../../target/qla2xxx/21:02:00:e0:8b:d1:67:2c/tpgt_1/lun/lun_0 /sys/kernel/config/target/qla2xxx/21:02:00:e0:8b:d1:67:2c/tpgt_1/acls/10:00:00:06:2b:11:3f:a0/lun_0/051a9e660f echo 0 > /sys/kernel/config/target/qla2xxx/21:02:00:e0:8b:d1:67:2c/tpgt_1/acls/10:00:00:06:2b:11:3f:a0/lun_0/write_protect #### Trigger to enable qla2xxx Target Portal Group echo 1 > /sys/kernel/config/target/qla2xxx/21:02:00:e0:8b:d1:67:2c/tpgt_1/enable When the machine boots on 3.9, I get the following messages in dmesg during target setup: [ 64.992066] Rounding down aligned max_sectors from 4294967295 to 4294967288 [ 66.268313] Rounding down aligned max_sectors from 4294967295 to 4294967288 [ 66.629656] emulate_write_cache cannot be changed when underlying HW reports WriteCacheEnabled, ignoring request [ 66.795981] Rounding down aligned max_sectors from 4294967295 to 4294967288 [ 66.954986] emulate_write_cache cannot be changed when underlying HW reports WriteCacheEnabled, ignoring request [ 67.037422] Rounding down aligned max_sectors from 4294967295 to 4294967288 [ 67.280278] emulate_write_cache cannot be changed when underlying HW reports WriteCacheEnabled, ignoring request [ 67.374967] Rounding down aligned max_sectors from 4294967295 to 4294967288 [ 67.613493] emulate_write_cache cannot be changed when underlying HW reports WriteCacheEnabled, ignoring request Please let me know how I can help to get this figured out. Cheers, Chris -- Chris Boot bootc@xxxxxxxxx -- To unsubscribe from this list: send the line "unsubscribe target-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
