From: Martin Wilck <mwilck@xxxxxxxx>
Hello Christophe, hello Ben,
The current multipathd unix listener code has various deficiencies.
- client disconnects aren't handled correctly,
- the uxsock_timeout is applied for receiving, handling, and
responding to the client requests separately, rather than for
the entire operation,
- timeouts are logged, but not acted upon, causing the timeout
to be noticed in the client rather than in the server.
- clients may see a timeout while "reconfigure" is running,
- unpriviledged (non-root) client connections don't work
correctly
- most importantly, the code busy-loops, polls, or waits in
various places in called subroutines, which is a no-go in a
piece of code designed as an event handler and may lead
to spurious timeouts and delayed reaction e.g. to signals
or client requests.
This patch set approaches all these issues. Fixing the last one,
in particular, requires a major refactoring of the uxlsnr code.
Overall, the reliability and latency of client request handling
and signal handling by multipathd should be noticeably improved
by this patch set.
The biggest problem (waiting for the vecs lock in a client handler)
can only be fixed by moving this wait into the handlers ppoll()
loop (another possible fix would have been to handle all clients
in separate threads, but that would have required even more
complexity). The patch set achieves this by adding an eventfd-based
notification mechanism to the vecs lock, which can be passed to
ppoll() to wake up when the lock is freed.
Furthermore, client requests can't be handled in a single poll
iteration any more. Therefore the client connection becomes stateful,
and is handled by a state machine using the states RECEIVE, PARSE,
WAIT FOR LOCK, WORK, and SEND.
The refactoring is done step by step for ease (hopefully) of
review. 1/35-4/35 add utility code that will be used by the uxlsnr
refactoring. 5/35-7/35 are some independent patches that
aren't directly related to uxlnsr, but fix issues that I observed
while working on this set. 14/35 too, but it deserves special mention
becaiuse it introduces a new option and changes the default behavior
of "multipathd reconfigure".
8/35-13/35 are minor fixups in the client handling code. This code is
strongly related to the uxlsnr, thus I thought I'd rather fix it
before making the other changes. In 25/35, the cli-handlers are
converted to use the strbuf API everywhere instead of separate "reply"
and "len" arguments. 15/35-18/35 are minor fixes for the
uxlsnr. 19/35-34/35 are the actual refactoring patches for the uxlsnr
code. First I move some code around unchanged, then I add the
state machine (handle_client()) and move the code into it piece
by piece. 35/35 adds a fix for the client side (multipathd -k).
CC'ing Lixiaokeng and Chongyun Wu, as they have test cases that use
the client code heavily AFAIR. Testing by 3rd parties would be
very welcome.
Comments welcome, regards,
Martin
Martin Wilck (35):
libmultipath: add timespeccmp() utility function
libmultipath: add trylock() helper
libmultipath: add optional wakeup functionality to lock.c
libmultipath: print: add __snprint_config()
libmultipath: improve cleanup of uevent queues on exit
multipathd: fix systemd notification when stopping while reloading
multipathd: improve delayed reconfigure
multipathd: cli.h: formatting improvements
multipathd: cli_del_map: fix reply for delayed action
multipathd: add prototype for cli_handler functions
multipathd: make all cli_handlers static
multipathd: add and set cli_handlers in a single step
multipathd: cli.c: use ESRCH for "command not found"
multipathd: add "force_reconfigure" option
multipathd: uxlsnr: avoid stalled clients during reconfigure
multipathd: uxlsnr: handle client HUP
multipathd: uxlsnr: use symbolic values for pollfd indices
multipathd: uxlsnr: avoid using fd -1 in ppoll()
multipathd: uxlsnr: data structure for stateful client connection
multipathd: move uxsock_trigger() to uxlsnr.c
multipathd: move parse_cmd() to uxlsnr.c
multipathd: uxlsnr: remove check_timeout()
multipathd: uxlsnr: move client handling to separate function
multipathd: uxlsnr: use main poll loop for receiving
multipathd: use strbuf in cli_handler functions
multipathd: uxlsnr: check root on connection startup
multipathd: uxlsnr: pass struct client to uxsock_trigger() and
parse_cmd()
multipathd: uxlsnr: move handler execution to separate function
multipathd: uxlsnr: use parser to determine non-root commands
multipathd: uxlsnr: merge uxsock_trigger() into state machine
multipathd: uxlsnr: add idle notification
multipathd: uxlsnr: add timeout handling
multipathd: uxlsnr: use poll loop for sending, too
multipathd: uxlsnr: drop client_lock
multipathd: uxclt: allow client mode for non-root, too
libmultipath/config.c | 1 +
libmultipath/config.h | 1 +
libmultipath/configure.c | 19 +-
libmultipath/defaults.h | 1 +
libmultipath/dict.c | 4 +
libmultipath/libmultipath.version | 15 +
libmultipath/lock.c | 12 +-
libmultipath/lock.h | 11 +-
libmultipath/print.c | 34 +-
libmultipath/print.h | 2 +
libmultipath/time-util.c | 12 +
libmultipath/time-util.h | 1 +
libmultipath/uevent.c | 49 ++-
multipath/multipath.8 | 6 +-
multipath/multipath.conf.5 | 17 +
multipathd/cli.c | 172 ++-------
multipathd/cli.h | 101 +++---
multipathd/cli_handlers.c | 553 ++++++++++++++---------------
multipathd/cli_handlers.h | 61 +---
multipathd/main.c | 238 +++++--------
multipathd/main.h | 3 +-
multipathd/multipathd.8 | 6 +-
multipathd/uxlsnr.c | 566 +++++++++++++++++++++++-------
multipathd/uxlsnr.h | 4 +-
24 files changed, 1043 insertions(+), 846 deletions(-)
--
2.33.0
--
dm-devel mailing list
dm-devel@xxxxxxxxxx
https://listman.redhat.com/mailman/listinfo/dm-devel
