Replying to myself, On Thu, Mar 28, 2024 at 05:17:18PM -0500, Eric Blake wrote: > On Thu, Mar 28, 2024 at 04:39:01PM -0400, Stefan Hajnoczi wrote: > > cp(1) and backup tools use llseek(SEEK_HOLE/SEEK_DATA) to skip holes in files. > > > > > In the block device world there are similar concepts to holes: > > - SCSI has Logical Block Provisioning where the "mapped" state would be > > considered data and other states would be considered holes. > > BIG caveat here: the SCSI spec does not necessarily guarantee that > unmapped regions read as all zeroes; compare the difference between > FALLOC_FL_ZERO_RANGE and FALLOC_FL_PUNCH_HOLE. While lseek(SEEK_HOLE) > on a regular file guarantees that future read() in that hole will see > NUL bytes, I'm not sure whether we want to make that guarantee for > block devices. This may be yet another case where we might want to > add new SEEK_* constants to the *seek() family of functions that lets > the caller indicate whether they want offsets that are guaranteed to > read as zero, vs. merely offsets that are not allocated but may or may > not read as zero. Skipping unallocated portions, even when you don't > know if the contents reliably read as zero, is still a useful goal in > some userspace programs. > > > - NBD has NBD_CMD_BLOCK_STATUS for querying whether blocks are present. The upstream NBD spec[1] took the time to represent two bits of information per extent, _because_ of the knowledge that not all SCSI devices with TRIM support actually guarantee a read of zeroes after trimming. That is, NBD chose to convey both: NBD_STATE_HOLE: 1<<0 if region is unallocated, 0 if region has not been trimmed NBD_STATE_ZERO: 1<<1 if region reads as zeroes, 0 if region contents might be nonzero it is always safe to describe an extent as value 0 (both bits clear), whether or not lseek(SEEK_DATA) returns the same offset; meanwhile, traditional lseek(SEEK_HOLE) on filesystems generally translates to a status of 3 (both bits set), but as it is (sometimes) possible to determine that allocated data still reads as zero, or that unallocated data may not necessarily read as zero, it is also possible to implement NBD servers that don't report both bits in parallel. If we are going to enhance llseek(2) to expose more information about underlying block devices, possibly by adding more SEEK_ constants for use in the entire family of *seek() API, it may be worth thinking about whether it is worth userspace being able to query this additional distinction between unallocated vs reads-as-zero. [1] https://github.com/NetworkBlockDevice/nbd/blob/master/doc/proto.md#baseallocation-metadata-context -- Eric Blake, Principal Software Engineer Red Hat, Inc. Virtualization: qemu.org | libguestfs.org
