Re: [PATCH 0/11] Update version of write stream ID patchset

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Mar 4, 2016, at 9:10 AM, Jens Axboe <axboe@xxxxxx> wrote:
> 
> It's been a while since I last posted the write stream ID patchset, but
> here is an updated version.
> 
> The original patchset was centered around the current NVMe streams
> proposal, but there was a number of issues with that. It's now in a
> much beter state, and hopefully will make it into 1.3 of the spec
> soon.
> 
> To quickly re-summarize the intent behind write stream IDs, it's to
> be able to provide a hint to the underlying storage device on what
> writes could feasibly be grouped together. If the device is able to
> group writes of similar life times on media, then we can greatly reduce
> the amount of data that needs to be copied around at garbage collection
> time. This gives us a better write amplification factor, which leads
> to better device life times and better (and more predictable)
> performance at steady state.

What are your thoughts on reserving a small number of the stream ID values
for filesystem metadata (e.g. the first 31 since 0 == unused)?  One of the
requests by several people at FAST was to be able to identify filesystem
metadata at the block layer for a variety of reasons (e.g. blktrace, etc).
I believe Ted said that Google is doing something similar for IO analysis.

For example, something like the following:

enum {
       SID_UNSET           = 0,
       SID_SUPERBLOCK      = 1,
       SID_ALLOCATIONGROUP = 2,
       SID_BLOCK_BITMAP    = 3,
       SID_INODE_BITMAP    = 4,
       SID_INODE           = 5,
       SID_INTERNAL_TREE   = 6,
       SID_DIRECTORY       = 7,
       SID_JOURNAL         = 8,
       SID_EXTENT          = 9,
       SID_XATTR           = 10,
       SID_DATA_FILE_4KB   = 11,
       SID_DATA_FILE_16KB  = 12,
       SID_DATA_FILE_64KB  = 13,
       SID_DATA_FILE_256KB = 14,
       SID_DATA_FILE_1MB   = 15,
       SID_DATA_FILE_4MB   = 16,
       SID_DATA_FILE_16MB  = 17,
       SID_DATA_FILE_64MB  = 18,
       SID_DATA_FILE_256MB = 19,
       SID_DATA_FILE_1GB   = 20,
       SID_DATA_FILE_LARGE = 21,
       SID_DATA_DIRECT     = 22,
       SID_LAST
};

though it would need to be expanded somewhat to include generic metadata
types from other filesystems.

Cheers, Andreas

> There's been a number of changes to this patchset since it was last
> posted. In summary:
> 
> 1) The bio parts have been bumped to carry 16 bits of stream data, up
>   from 8 and 12 in the original series.
> 
> 2) Since the interface grew some more options, I've moved away from
>   fadvise and instead added a new system call. I don't feel strongly
>   about what interface we use here, another option would be to have a
>   (big) set of fcntl() commands instead.
> 
> 3) The kernel now manages the ID space, since we have moved to a host
>   assigned model. This is done on a backing_dev_info basis, and the
>   btrfs patch has been updated to show how this can be used for nested
>   devices on btrfs/md/dm/etc. This could be moved to the request queue
>   as well, again I don't feel too strongly aboout this specific part.
> 
> Those are the big changes.
> 
> The patches are against Linus' current -git tip.
> 
> --
> Jens Axboe
> 
> 


Cheers, Andreas





Attachment: signature.asc
Description: Message signed with OpenPGP using GPGMail


[Index of Archives]     [Linux Ext4 Filesystem]     [Union Filesystem]     [Filesystem Testing]     [Ceph Users]     [Ecryptfs]     [AutoFS]     [Kernel Newbies]     [Share Photos]     [Security]     [Netfilter]     [Bugtraq]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux Cachefs]     [Reiser Filesystem]     [Linux RAID]     [Samba]     [Device Mapper]     [CEPH Development]
  Powered by Linux