Am Samstag, 14. April 2012, 14:37:47 schrieb Nicholas A. Bellinger: > On Sat, 2012-04-14 at 18:35 +0200, Henning Becker wrote: > > Am Dienstag, 10. April 2012, 23:54:06 schrieb Nicholas A. Bellinger: > > > On Mon, 2012-04-09 at 23:26 +0200, Henning Becker wrote: > <SNIP> > > > > Hi Henning, > > > > > > Thanks additional for the feedback on your setup. > > > > > > > Should I test with ACLs enabled? > > > > > > At this point I'm still thinking it's something to do with active I/O > > > shutdown with sessions attached to dynamic ACLs with demo-mode > > > (generate_node_acls=1) enabled.. > > > > Hi Nicholas, > > I have bad news. The problem just occured also with generate_node_acls=0. > > > > The Oops happend twice this time with slighty different stack traces. > > Perhaps, it can help you: http://pastebin.com/jpnuYxqV > > Hi Henning, > > Ok, I think I've identified the cause of this oops within iscsi-target. > > It has to do with the ordering in which your scripts are tearing down > the configfs layout. Looking at the inotify log again I see the > following ordering: > > # Tear down LUN=0 from TPG=1 > /sys/kernel/config/target/iscsi/iqn.2012-04.lan.storage:iscsi.storage/tpgt_1 > /lun/lun_0/ DELETE,ISDIR statistics > /sys/kernel/config/target/iscsi/iqn.2012-04.lan.storage:iscsi.storage/tpgt_ > 1/lun/lun_0/ DELETE_SELF > /sys/kernel/config/target/iscsi/iqn.2012-04.lan.storage:iscsi.storage/tpgt_ > 1/lun/ DELETE,ISDIR lun_0 > > # Release IBLOCK backend device > /sys/kernel/config/target/core/iblock_0/iscsiLUNTest/ DELETE_SELF > /sys/kernel/config/target/core/iblock_0/ DELETE,ISDIR iscsiLUNTest > > # Echo '0 > enable' to disable TPG > /sys/kernel/config/target/iscsi/iqn.2012-04.lan.storage:iscsi.storage/ > CLOSE_NOWRITE,CLOSE,ISDIR > /sys/kernel/config/target/iscsi/iqn.2012-04.lan.storage:iscsi.storage/tpgt_ > 1/ MODIFY enable > /sys/kernel/config/target/iscsi/iqn.2012-04.lan.storage:iscsi.storage/tpgt_ > 1/ OPEN enable > > So it appears with your custom scripts that the LUN=0 + IBLOCK backend > is being released *before* explicitly disabling the TPG and forcing all > of the active sessions to shutdown. > > The OOPs itself is being caused by the removal of the IBLOCK backend, as > there is code in the iscsi_cmd descriptor release path that depends upon > the backend being in place (although removing the TPG LUN is OK).. This > is a genuine bug, for which I'll need to think some more to best resolve > in order to avoid extra overhead within the existing data I/O fast > path.. > > That said, the work-around for this bug is to change your custom scripts > to follow what rtslib/lio-utils currently does for TPG removal. That > is: > > 1: Echo '0 > enable' to disable TPG > 2: Tear down NodeACLs+MappedLUNs from TPG > 3: Tear down LUN from TPG > 4: Tear down entire TPG > 4: Release IBLOCK backend device Hi Nicholas, I'm just running my cluster according to your specs for 3 weeks now and the problem has not occured anymore. :-) > > I'm quite certain this will avoid the bug in question by forcing > shutdown of all active sessions at step #1, instead of doing this part > at the end of the sequence as done in your current setup. > > Please give it a shot and let me know if you have problems getting your > scripts to sync with what the official userspace code is doing here. Which official userspace code does that? I'm currently just calling lio_node and it didn't refuse me, to release an iblock which is still connected to a portal. Thanks, Henning > > Thanks, > > --nab -- To unsubscribe from this list: send the line "unsubscribe target-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
