On 03/04/2016 02:01 PM, Jeff Moyer wrote:
Jens Axboe <axboe@xxxxxx> writes:
On 03/04/2016 12:42 PM, Jeff Moyer wrote:
My main question is why expose this to userspace at all? If we're
keeping track of write streams per file, then why not implement that in
the kernel, transparent to the application? That would benefit all
applications instead of requiring application developers to opt in.
Because lots of different files could be the same write ID.
Do you mean that the application will want to have multiple files that
share the same write ID, or that there will be collisions due to the
small ID space (I think the former)? There's no way to avoid the
latter, of course. For the former, I agree that encoding a per-file
policy in the kernel could be limiting.
The former - and it won't just be limiting, it'll be mostly useless...
It's not like we're going to have millions of streams available, you
have to group them more wisely. Unless the policy is
one-stream-per-file always, then we can't put that sort of thing in
the kernel. The kernel has no way of knowing.
I know that hard-coding a one stream per file (or directory) scheme
wouldn't be the most ideal situation, but I wonder if it covers 90% of
the use cases without requiring application involvement. Some numbers
here would be very useful in supporting one scheme versus another.
It'd be even harder to convince applications to change their file
layout, and it'd be a more involved change than simply opting in to add
a system call to set a stream ID when you open a file. It's really not a
hard change at all.
I'm sure your argument will have something to do with how stream id's
are allocated/freed (expensive/slow, limited resource, whatever), but
that really just gets back to Martin's original questions about what we
should expect from the hardware and what the programming model should
look like (questions that are, afaik, still open).
That's orthogonal, really. The open/close might be expensive, or it
might not be, it has no real bearing on how you assign specific writes
to specific stream IDs.
It may be important if you plan to open and close the streams often.
Of course, I'm saying that the fact that if it's expensive or not, it
has no impact on that part of the discussion.
Again, it's not clear to me what the hardware supports or requires in
this area, so I'm not sure if it's a relevant question. -ENOSPEC. :)
I'm not against write streams, I think it's a neat idea. I just think
it will die on the vine if you require application developers to opt
in. Not all storage is SSDs, and I don't like that SSDs now have to be
treated differently by the programmer.
But that's why it's kept really simple. There are people that want to
make this more involved, and tie QoS criteria to streams. My argument
there has been what you are saying, it will never be used or get
adopted. For streams in general, the wins are big enough that
applications will care. And it's not difficult to use at all...
I'm not convinced applications will care. :) How many developers will
even know that they should use this interface?
You have to market it a bit, obviously. But if you consume flash
devices, then you WILL be interested in cutting your write amplification
down a lot. Especially if you have tons of them.
So applications might not care, the people that use them certainly will.
And most people care about performance, too.
What I'm saying is that I disagree wildly. There's enough of a carrot
here for people to care.
It's not just SSDs, either. Could be used for tiered storage in
general. That would mostly require going a bit further and assigning
performance characteristics to specific stream IDs, but there's
nothing preventing that from being done down the road. For now, this
is just a basic interface with a kernel managed stream ID space
attached.
OK. I'm still of the opinion that we should try to make this
transparent. I could be swayed by workload descriptions and numbers
comparing approaches, though.
You can't just waive that flag and not have a solution. Any solution in
that space would imply having policy in the kernel. A "just use a stream
per file" is never going to work.
--
Jens Axboe
--
To unsubscribe from this list: send the line "unsubscribe linux-block" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
