Hi Ilias,
Thanks for your detailed email. Comments are inline below.
On Wed, 2014-12-10 at 18:36 +0200, Ilias Tsitsimpis wrote:
> Hi all,
>
> I am trying out Linux SCSI Target and it seems that the way persistent
> reservations are handled does not conform with SPC-4.
> SPC4r17 (5.7.10) states that:
>
> The persistent reservation holder is determined by the type of the
> persistent reservation as follows:
> a) For a persistent reservation of the type Write Exclusive – All
> Registrants or Exclusive Access – All Registrants, the persistent
> reservation holder is any registered I_T nexus;
>
> While Table 51 states that:
>
> 'Write Exclusive – All Registrants' or 'Exclusive Access – All
> Registrants' reservations shall be released when the registration
> for the last registered I_T nexus is removed or when the type or
> scope is changed.
>
> This doesn't seem to be the case with Linux SCSI Target. More
> particularly, I have set up an iSCSI target, using the Linux SCSI Target
> implementation, and I am connecting to it through two different VMs
> using the open-iscsi project.
>
> root@node1:~# sg_inq /dev/sda
> standard INQUIRY:
> PQual=0 Device_type=0 RMB=0 version=0x05 [SPC-3]
> [AERC=0] [TrmTsk=0] NormACA=0 HiSUP=0 Resp_data_format=2
> SCCS=1 ACC=0 TPGS=3 3PC=1 Protect=0 BQue=0
> EncServ=0 MultiP=0 [MChngr=0] [ACKREQQ=0] Addr16=0
> [RelAdr=0] WBus16=0 Sync=0 Linked=0 [TranDis=0] CmdQue=1
> length=36 (0x24) Peripheral device type: disk
> Vendor identification: LIO-ORG
> Product identification: FILEIO
> Product revision level: 4.0
> Unit serial number: 9efc01f5-429d-42be-b507-9dc4439f511d
>
> root@node2:~# sg_inq /dev/sda
> standard INQUIRY:
> PQual=0 Device_type=0 RMB=0 version=0x05 [SPC-3]
> [AERC=0] [TrmTsk=0] NormACA=0 HiSUP=0 Resp_data_format=2
> SCCS=1 ACC=0 TPGS=3 3PC=1 Protect=0 BQue=0
> EncServ=0 MultiP=0 [MChngr=0] [ACKREQQ=0] Addr16=0
> [RelAdr=0] WBus16=0 Sync=0 Linked=0 [TranDis=0] CmdQue=1
> length=36 (0x24) Peripheral device type: disk
> Vendor identification: LIO-ORG
> Product identification: FILEIO
> Product revision level: 4.0
> Unit serial number: 9efc01f5-429d-42be-b507-9dc4439f511d
>
>
> I register both of the IT_nexuses with the target.
>
> root@node1:~# sg_persist --out --register --param-sark 0x1 /dev/sda
> root@node2:~# sg_persist --out --register --param-sark 0x2 /dev/sda
>
> root@node1:~# sg_persist --in --read-full-status /dev/sda
> LIO-ORG FILEIO 4.0
> Peripheral device type: disk
> PR generation=0x2
> Key=0x1
> All target ports bit clear
> Relative port address: 0x1
> not reservation holder
> Transport Id of initiator:
> iSCSI name and session id: iqn.1993-08.org.debian:01:node1
> Key=0x2
> All target ports bit clear
> Relative port address: 0x1
> not reservation holder
> Transport Id of initiator:
> iSCSI name and session id: iqn.1993-08.org.debian:01:node2
>
Looks fine.
> Then I get a persistent reservation with type 'Exclusive Access - All
> Registrants' for node1.
>
> root@node1:~# sg_persist --out --reserve --prout-type=8 --param-rk=0x1 /dev/sda
> root@node1:~# sg_persist --in --read-full-status /dev/sda
> LIO-ORG FILEIO 4.0
> Peripheral device type: disk
> PR generation=0x2
> Key=0x1
> All target ports bit clear
> Relative port address: 0x1
> << Reservation holder >>
> scope: LU_SCOPE, type: Exclusive Access, all registrants
> Transport Id of initiator:
> iSCSI name and session id: iqn.1993-08.org.debian:01:node1
> Key=0x2
> All target ports bit clear
> Relative port address: 0x1
> not reservation holder
> Transport Id of initiator:
> iSCSI name and session id: iqn.1993-08.org.debian:01:node2
>
> The node2 is listed as "not reservation holder".
> Shouldn't node2 be displayed as a Reservation holder since the
> persistent reservation is of type 'Exclusive Access, all registrants'?
>
Good question. So from the perspective of CDB filtering logic in
drivers/target/target_core_pr.c:core_scsi3_pr_seq_non_holder(),
the /dev/sdb device above (registration active, not explicit reservation
holder) will still be treated like a implicit reservation holder with an
all registrants type reservation:
Specifically the code here:
switch (pr_reg_type) {
.......
case PR_TYPE_WRITE_EXCLUSIVE_ALLREG:
we = 1;
case PR_TYPE_EXCLUSIVE_ACCESS_ALLREG:
/*
* Each registered I_T Nexus is a reservation holder.
*/
all_reg = 1;
if ((se_deve->def_pr_registered) && !(ignore_reg))
registered_nexus = 1;
break;
and further down here:
if (we && !registered_nexus) {
..............
} else if ((reg_only) || (all_reg)) {
if (registered_nexus) {
/*
* For PR_*_REG_ONLY and PR_*_ALL_REG reservations,
* allow commands from registered nexuses.
*/
pr_debug("Allowing implicit CDB: 0x%02x for %s"
" reservation\n", cdb[0],
core_scsi3_pr_dump_type(pr_reg_type));
return 0;
}
}
ensures that when another I_T nexus takes an explicit reservation with
an all registrants type reservation (eg: /dev/sda), that all of the
implicit reservation holders (eg: /dev/sdb) are treated the same.
So from the perspective of the READ_FULL_STATUS output above, my
original interpretation was that the R_HOLDER is intended to display the
explicit reservation holder obtained through a RESERVE service action,
and not the implicit reservation holders.
After looking at SPC-4 again, I don't see any language about R_HOLDER +
being specific to explicit RESERVEs or implicit all registrants
reservations.
> If I then proceed to unregister node1, the reservation gets released
> altogether. Shouldn't the reservation remain, until the last registered
> I_T nexus goes away? I think this violates SPC-4, please see the excerpt
> above.
>
> root@node1:~# sg_persist --out --register --param-rk=0x1 --param-sark=0x0 /dev/sda
> root@node1:~# sg_persist --in --read-full-status /dev/sda
> LIO-ORG FILEIO 4.0
> Peripheral device type: disk
> PR generation=0x2
> Key=0x2
> All target ports bit clear
> Relative port address: 0x1
> not reservation holder
> Transport Id of initiator:
> iSCSI name and session id: iqn.1993-08.org.debian:01:node2
>
> root@node2:~# sg_persist --in --read-reservation /dev/sda
> LIO-ORG FILEIO 4.0
> Peripheral device type: disk
> PR generation=0x2, there is NO reservation held
>
> Is this a bug or am I missing something?
>
One requirement about the current implementation of all registrants
logic is that it requires one explicit reservation to be in place, for
all of the other registrations to be treated as implicit reservations as
per the spec.
This is because as you might have noticed above, the reservation type to
declare all registrants is not indicated at registration time, but at
explicit reservation time.
So given the ambiguity in the spec in this regard, I think this would be
a great question for the T10 list. I'll try to articulate the points
above, and send out an email to them shortly for clarification.
Thanks for reporting!
--nab
--
To unsubscribe from this list: send the line "unsubscribe target-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
