Jens Axboe <axboe@xxxxxx> writes: > On 03/04/2016 12:42 PM, Jeff Moyer wrote: >> My main question is why expose this to userspace at all? If we're >> keeping track of write streams per file, then why not implement that in >> the kernel, transparent to the application? That would benefit all >> applications instead of requiring application developers to opt in. > > Because lots of different files could be the same write ID. Do you mean that the application will want to have multiple files that share the same write ID, or that there will be collisions due to the small ID space (I think the former)? There's no way to avoid the latter, of course. For the former, I agree that encoding a per-file policy in the kernel could be limiting. > It's not like we're going to have millions of streams available, you > have to group them more wisely. Unless the policy is > one-stream-per-file always, then we can't put that sort of thing in > the kernel. The kernel has no way of knowing. I know that hard-coding a one stream per file (or directory) scheme wouldn't be the most ideal situation, but I wonder if it covers 90% of the use cases without requiring application involvement. Some numbers here would be very useful in supporting one scheme versus another. >> I'm sure your argument will have something to do with how stream id's >> are allocated/freed (expensive/slow, limited resource, whatever), but >> that really just gets back to Martin's original questions about what we >> should expect from the hardware and what the programming model should >> look like (questions that are, afaik, still open). > > That's orthogonal, really. The open/close might be expensive, or it > might not be, it has no real bearing on how you assign specific writes > to specific stream IDs. It may be important if you plan to open and close the streams often. Again, it's not clear to me what the hardware supports or requires in this area, so I'm not sure if it's a relevant question. -ENOSPEC. :) >> I'm not against write streams, I think it's a neat idea. I just think >> it will die on the vine if you require application developers to opt >> in. Not all storage is SSDs, and I don't like that SSDs now have to be >> treated differently by the programmer. > > But that's why it's kept really simple. There are people that want to > make this more involved, and tie QoS criteria to streams. My argument > there has been what you are saying, it will never be used or get > adopted. For streams in general, the wins are big enough that > applications will care. And it's not difficult to use at all... I'm not convinced applications will care. :) How many developers will even know that they should use this interface? > It's not just SSDs, either. Could be used for tiered storage in > general. That would mostly require going a bit further and assigning > performance characteristics to specific stream IDs, but there's > nothing preventing that from being done down the road. For now, this > is just a basic interface with a kernel managed stream ID space > attached. OK. I'm still of the opinion that we should try to make this transparent. I could be swayed by workload descriptions and numbers comparing approaches, though. Cheers, Jeff -- To unsubscribe from this list: send the line "unsubscribe linux-block" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
