Hi Martin, I'm still thinking through this, but I wanted to respond so you know I'm not dropping the dialogue. I understand complications you face trying to support a variety of drive implementations that evolve different capabilities over multiple generations. We face the same issues. In some cases we don't have the flexibility to accommodate everything though, and we have to try to drive the ecosystem towards more consistency, typically though participation in standards development. I recognize the requirement to support ndob=1 for WS(unmap=1) is tied to SBC Base 2016. I can't say you can rely on that from our RAID solution because I haven't scoped what all else conforming to SBC Base 2016 entails. Supporting RSOC seems like a more tractable solution, but I still need to scope that as well. I didn't realize RSOC one_command format allows reporting which fields in the CDB are supported. Now I see that it does, and that it can be used to indicate support for the ndob bit. It may be worth implementing support for RSOC if that prompts a future version of Linux that recognizes it and uses NDOB accordingly. I really appreciate your feedback. Thanks, Bob Sheffield - Broadcom, Inc. -----Original Message----- From: Martin K. Petersen [mailto:martin.petersen@xxxxxxxxxx] Sent: Wednesday, February 27, 2019 8:53 PM To: Bob Sheffield <bob.sheffield@xxxxxxxxxxxx> Cc: Kashyap Desai <kashyap.desai@xxxxxxxxxxxx>; Martin K. Petersen <martin.petersen@xxxxxxxxxx>; Christoph Hellwig <hch@xxxxxx>; linux-scsi <linux-scsi@xxxxxxxxxxxxxxx> Subject: Re: [scsi] write same with NDOB (no data-out buffer) support Hi Bob, > First, we don't know what pattern the host will generate. And the > basic premise of ndob=0 suggests that the pattern provided by the host > may not be all zeros. I can assure you that UNMAP=1 and NDOB=0 in the Linux sd driver case will be all zeroes in the data-out buffer. But I have no control over sg passthrough, obviously. > But the RAID controller maps host data to multiple drives, and > effective unmapping of blocks on actual media requires consistency > across the attached drives. The only reasonable assumption/requirement > is that the drives all implement an all-zeros provisioning > initialization pattern. Yep. I also think it's important to emphasize that we have several block layer operations that may result in a WRITE SAME(16) command being issued: - The user may ask to deallocate blocks. In that case we don't care about the block contents for subsequent READs. If the device supports the UNMAP command, we will use that. Otherwise we fall back to WRITE SAME(16) or (10) with UNMAP set. - The user may ask to zero blocks. And for that use case we only use WRITE SAME(16) or (10) with a zeroed data-out buffer. The UNMAP flag is set depending on user preference wrt. block deallocation and the device-reported LBPRZ flag. - The user may ask to set blocks to a non-zero pattern. And in that case we'll also use WRITE SAME(16) or (10) but without UNMAP set. Also, we have to support a wide range of LBP implementations. There were many, many iterations in SBC before the spec was finalized. It was a constantly moving target. Consequently, we support devices that predate LBPU, LBPWS, and LBPWS10 being added to the Block Provisioning VPD. We also support devices that predate the Block Provisioning VPD existing and which are keyed off of UNMAP and WRITE SAME limits in the Block Limits VPD. And for devices with the older, shorter Block Limits VPD that didn't have UNMAP and WRITE SAME limits, we still work if LBPME is set. We even support thin provisioned devices that predate the LBPME (formerly TPE) bit in READ CAPACITY(16), although it requires admin intervention to turn this on. So it's not as clear cut as saying that SBC-4 requires this or that. I have a decade of different implementations to support. > SATA or NVMe which do not define commands equivalent to WRITE SAME > (unmap=1, ndob=0) with a non-zero data-out buffer. In NVMe there's WRITE ZEROES w/ DEAC. > Bottom line is that SCSI defined WRITE SAME (unmap=1, ndob=1) as it > did (with the requirement that support for unmap=1 implies support for > ndob=1) for good reason. I understand the desire on your end to avoid the compare. We currently don't support the feature set VPD page because industry adoption has been glacially slow. I can look into keying off of SBC Base 2016 if that's your preferred option but I'd rather use RSOC if you support that since it's much more widely implemented... -- Martin K. Petersen Oracle Linux Engineering
