On 1/11/25 12:57, Bird, Tim wrote:
Hey Rob, This is a great review of /dev, /sys and the different
ways that /dev gets populated.
Feel free to link stuff from wikis or some such. The newest of those
documents was written in 2007.
For a lot of embedded Linux devices, the only bus where
new items can show up dynamically is USB.
Yup, /sys/bus/usb/devices is in there too and when a driver binds to
them, they wind up in /sys/block and such as well. (you USED to have to
seprately mount a usbfs under /sys but they finally acknowledged that
was silly about 5 years ago, hence
https://askubuntu.com/questions/1218321/if-usbfs-has-been-deprecated-then-why-is-sys-bus-usb-drivers-usbfs-directory-p)
When a driver DOESN'T automatically bind to them it gets a bit
complicated, and one of the things mdev can be configured to do is act
as a firmware loader! Which is just... Ahem, there are YEARS of poor
design decisions the kernel guys made, where they ignored a mechanism
they already had an implemented something more complicated. The
mechanism whereby the kernel opens a firmware file and read it directly
out of the filesystem instead of calling a hotplug helper was... I'm
just going to gloss over that.
Anyway, I saw patches go by ala
https://patchwork.ozlabs.org/project/buildroot/patch/1436188175-7912-1-git-send-email-luca@xxxxxxxxxxxxxxxx/
which says it's from 2015. I haven't really tried to do it because I
often know how the plumbing works and usually just implement a ten line
hack rather than looking up how to configure the more generic tool.
(Even the generic tool grew out of something I wrote 10 years earlier...)
Anyway, the kernel's "request module" plumbing tends to do a "give me
usb-vendorID-deviceID thing, and then there's alias plumbing lookup that
figures out what module name to insmod for that, and at various points
I've seen said alias lookup plumbing A) in the kernel, B) in module
headers, C) in modprobe config files under /etc or /lib or something.
*shrug* I build static kernels when given a choice, and don't source
hardware that needs drivers WITHOUT built-in firmware, so I am WAY out
of date on that stuff. I remember enough to look it up but not the
details off the top of my head. And I say that as someone who really
SHOULD care more:
http://lists.landley.net/pipermail/toybox-landley.net/2024-October/030549.html
It's on the todo list...
I've always thought the best solution (in terms of boot time)
was to use static nodes in /dev during early boot (that is, just
mknod the /dev nodes in the rootfs manually, and have them be present
before the kernel even runs). No dynamic discovery or boot-time
population of /dev needed.
The system knows what devices are available. While you can mknod a
major:minor node the kernel doesn't have a driver for, if you open it
you get some sort of -EWTF where the kernel goes "nope".
The kernel has a CONFIG_DEVTMPFS_MOUNT that automatically mounts
devtmpfs on /dev, but for SOME reason it doesn't apply to initramfs.
I've been irregularly posting a patch to MAKE it apply on and off for
most of a decade now:
https://lkml.iu.edu/hypermail/linux/kernel/2005.1/09399.html
And it's part of my mkroot kernel patches:
https://landley.net/bin/mkroot/latest/linux-patches/0003-Wire-up-CONFIG_DEVTMPFS_MOUNT-to-initramfs.patch
But Greg KH. Oh well...
Then, sometime later, use either
mdev or devtmpfs to accumulate (and remove?) other
runtime-plugged devices. This can be done after the time-critical
phase of booting.
Does this overall approach work, or is there some in-kernel
connections that may be missing if the dynamic tools are
not used from startup?
I mean it more or less works, it's just... pointless manual maintenance
of something the kernel does for you in a very small amount of code? (In
devtmpfs, the /dev node being there means something. In a static /dev,
it doesn't.)
So: I blather on a lot about my mkroot project, which is a 400 line bash
script that builds tiny linux system that boots to shell prompt (mostly
under qemu) on a dozen different architectures. And the init script in
that starts here:
https://github.com/landley/toybox/blob/master/mkroot/mkroot.sh#L102
And you see how the script does this setup (which is only needed when
you don't apply my patch to the kernel):
if ! mountpoint -q dev; then
mount -t devtmpfs dev dev
[ $$ -eq 1 ] && ! 2>/dev/null <0 && exec 0<>/dev/console 1>&0 2>&1
for i in ,fd /0,stdin /1,stdout /2,stderr
do ln -sf /proc/self/fd${i/,*/} dev/${i/*,/}; done
mkdir -p dev/shm
chmod +t /dev/shm
fi
(Don't ask me why devtmfs doesn't automatically have a "shm" directory
with the sticky bit, it's a subclass of tmpfs so it does the right thing
when it's there. And the middle two lines are just making
/dev/{stdin,stdout,stderr} because toysh doesn't special case /dev/stdin
like bash does so needs it in the filesystem if you're gonna use that.)
But that exec redirect line hit a bug, because when you don't have
devtmpfs automounting but ALSO don't have /dev/console in your
statically linked initramfs image... oh here, I explained it when I
fixed it:
https://github.com/landley/toybox/commit/0b2d5c2bb3f1
https://landley.net/notes-2024.html#06-08-2024
tl;dr in usr/main.c the kernel tries to open /dev/console and fails if
it's not already there, so PID 1 starts with stdin, stdout, and stderr
closed, so if something goes wrong in early boot init can't tell you why
it failed. This ONLY happens with static initramfs (created with cpio as
a normal user and you can't mknod without root access), because of
COURSE the kernel has two different codepaths for static vs dynamic, and
the dynamic one does a manual fixup for this issue. No really!
https://github.com/torvalds/linux/blob/master/init/noinitramfs.c#L18
And of course the code doing the fixup runs for ANY system that doesn't
have a static linked initramfs, including one where the bootloader
points it at an EXTERNAL cpio.gz image like qemu -initrd blah.cpio.gz,
because it checks for the external one AFTER doing the fixup. (Pop quiz:
if you have static _and_ external initrd, and they include the same
file, does the old one prevent the new one from extracting, does the new
one replace the old one, or does the new one APPEND to the old one? At
various points in history, it's done ALL THREE! I forget which is
current, replace I think?) So external initramfs and built-in initramfs
behave DIFFERENTLY in a subtle sharp edge ath I've complained at them
about for YEARS...
*shrug* I make this stuff work in as simple a way as I know how. Been
doing it for an embarrassingly long time now. But you _reach_ simple by
process of elimination, and proving a negative is a lot of work.
Thanks,
-- Tim
Rob
