On 4/27/22 02:03, Jakub Kicinski wrote:
On Tue, 26 Apr 2022 17:58:39 +0200 Hannes Reinecke wrote:
Plus there are more protocols being actively worked on (QUIC, PSP etc.)
Having per ULP special sauce to invoke a user space helper is not the
paradigm we chose, and the time as inopportune as ever to change that.
Which is precisely what we hope to discuss at LSF.
(Yes, I know, probably not the best venue to discuss network stuff ...)
Indeed.
Each approach has its drawbacks:
- Establishing sockets from userspace will cause issues during
reconnection, as then someone (aka the kernel) will have to inform
userspace that a new connection will need to be established.
(And that has to happen while the root filesystem is potentially
inaccessible, so you can't just call arbitrary commands here)
(Especially call_usermodehelper() is out of the game)
Indeed, we may need _some_ form of a notification mechanism and that's
okay. Can be a (more generic) socket, can be something based on existing
network storage APIs (IDK what you have there).
Which is one of the issues; we don't have any.
Network storage is accessed from userspace via read()/write(), and for
control everyone rolls its own thing.
So speaking of a 'network storage API' is quite optimistic.
My thinking was that establishing the session in user space would be
easiest. We wouldn't need all the special getsockopt()s which AFAIU
work around part of the handshake being done in the kernel, and which,
I hope we can agree, are not beautiful.
Well; that is open to debate.
During open-iscsi development (which followed your model of having a
control- and dataplane split between userspace and kernel) we found that
the model is great for keeping the in-kernel code simple.
But we also found that it's not so great once you come down to the
gritty details; you have to duplicate quite some protocol handling in
userspace, and you have to worry about session re-establishment.
Up to the point where we start wondering if moving things into userspace
was a good idea at all...
As for this particular interface: the problem we're facing is that TLS
has to be started in-band. For some reason NVMexpress decided not to
follow the traditional method of establishing the socket, start TLS, and
then start protocol processing, but it rather has specified a hybrid
model: establishing the socket, doing authentication, _then_ start TLS,
and only then continue with protocol processing on the TLS socket.
Moving _that_ into userspace would require us to move the most of the
protocol logic into userspace, too. Which really is something we want to
avoid, as this would be quite a code duplication.
Not to mention the maintenance burden, as issues would need to be fixed
in two locations.
(Talk to me about iscsiuio.)
And that's before we even get to things like TLS session tickets, which
opens yet another can of worms.
- Having ULP helpers (as with this design) mitigates that problem
somewhat in the sense that you can mlock() that daemon and having it
polling on an intermediate socket; that solves the notification problem.
But you have to have ULP special sauce here to make it work.
TBH I don't see how this is much different to option 1 in terms of
constraints & requirements on the user space agent. We can implement
option 1 over a socket-like interface, too, and that'll carry
notifications all the same.
- Moving everything in kernel is ... possible. But then you have yet
another security-relevant piece of code in the kernel which needs to be
audited, CVEd etc. Not to mention the usual policy discussion whether it
really belongs into the kernel.
Yeah, if that gets posted it'd be great if it includes removing me from
the TLS maintainers 'cause I want to sleep at night ;)
So I don't really see any obvious way to go; best we can do is to pick
the least ugly :-(
True, I'm sure we can find some middle ground between 1 and 2.
Preferably implemented in a way where the mechanism is separated
from the fact it's carrying TLS handshake requests, so that it can
carry something else tomorrow.
Which was actually our goal.
The whole thing started off with the problem on _how_ sockets could be
passed between kernel and userspace and vice versa.
While there is fd passing between processes via AF_UNIX, there is no
such mechanism between kernel and userspace.
So accept() was an easy way out as the implementation was quite simple.
And it was moved into net/tls as this was the primary use-case.
But it's by no means meant to be exclusive for TLS; we could expand it
to other things once there is a need.
But we really didn't want to over-engineer things here.
However, if you have another idea, by all means, do tell.
It's just that I don't think creating sockets in userspace is a great
fit for us.
Cheers,
Hannes
