The graph from my v3 patch series [0] which tries to resolve the virtual
memory lost bytes due to duplicates says it all:
+----------------------------------------------------------------------------+
14GB |-+ + + + + *+ +-|
| **** |
| *** |
| ** |
12GB |-+ ** +-|
| ** |
| ** |
| ** |
| ** |
10GB |-+ ** +-|
| ** |
| ** |
| ** |
8GB |-+ ** +-|
waste | ** ### |
| ** #### |
| ** ####### |
6GB |-+ **** #### +-|
| * #### |
| * #### |
| ***** #### |
4GB |-+ ** #### +-|
| ** #### |
| ** #### |
| ** #### |
2GB |-+ ** ##### +-|
| * #### |
| * #### Before ******* |
| **## + + + + After ####### |
+----------------------------------------------------------------------------+
0 50 100 150 200 250 300
CPUs count
So we really need to debug to see WTF, because really, WTF. The first
patch tries to answer the question if the issue is module auto-loading
being abused and that causing the issues. The patch proves that the
answer is no, but it does also help us find *a few* requests which can
get a bit of love to avoid duplicates. My system at least found one. So
it adds a debugging facility to let you do that.
As I was writing the commit log for my first patch series [0] I was noting
that this is it... and the obvious conclusion is that the culprit is udev
issuing requests per CPU for tons of modules. I didn't feel comfortable in
writing that this is it and we can't really do anything before really
trying hard. So I gave it a good 'ol college try. At first I wondered if
we could use file descriptor hints to just exlude users early on boot
before SYSTEM_RUNNING. I couldn't find much, but if there are some ways
to do that -- then the last patch can be simplified to do just that.
The second patch proves essentially that we can just send -EBUSY to
duplicate requests, at least for duplicate module loads and the world
doesn't fall apart. It *would* solve the issue. The patch however
borrows tons of the code from the first, and if we're realy going to
rely on something like that we may as well share. But I'm hopeful that
perhaps there are some jucier file descriptor tricks we can use to
just make a file mutually exlusivive and introduce a new kread which
lets finit_module() use that. The saving grace is that at least all
finit_module() calls *wait*, contray to request_module() calls and so
the solution can be much simpler.
The end result is 0 wasted virtual memory bytes.
Any ideas how not to make patch 2 suck as-is ?
Yes -- we can also go fix udev, or libkmod, and that's what should be
done. However, it seems silly to not fix if the fix is as trivial as
patch 2 demonstrates.
If you want to test / muck with all this you can use my branch
20230413-module-alloc-opts [1]:
[0] https://lkml.kernel.org/r/20230414050836.1984746-1-mcgrof@xxxxxxxxxx
[1] https://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/linux.git/log/?h=20230413-module-alloc-opts
Luis Chamberlain (2):
module: add debugging auto-load duplicate module support
kread: avoid duplicates
fs/kernel_read_file.c | 150 +++++++++++++++++++++++++
kernel/module/Kconfig | 40 +++++++
kernel/module/Makefile | 1 +
kernel/module/dups.c | 234 +++++++++++++++++++++++++++++++++++++++
kernel/module/internal.h | 15 +++
kernel/module/kmod.c | 23 +++-
kernel/module/main.c | 6 +-
7 files changed, 463 insertions(+), 6 deletions(-)
create mode 100644 kernel/module/dups.c
--
2.39.2
