On Wed, Jun 29, 2016 at 6:03 AM, Daniel P. Berrange <berrange@xxxxxxxxxx> wrote: > For those who aren't familiar, QEMU actually provides two completely > different sets of emulators > > - system emulators - they emulate a full virtual machine and thus run > a full guest OS. > - user emulators - they emulate the Linux userspace ABI letting you > run non-native arch executables directly. > > The user emulators are what I'm concerned with in this mail, so ignore > the system emulators. > > Currently all the user emulators are provided in the "qemu-user" RPM > which also includes files in /usr/lib/binfmt.d to register each emulator > binary as a binary format handler for its respective architecture. > > This is ok if you have a non-native arch binary that's statically linked > and you just want to run it from context of your main OS root filesystem. > Running dynamic linked binaries won't fly because if say running an arm > binary on x86_64 host, it'll look for /lib/libc.so and find the i386 one, > instead of the arm one. You can't set LD_LIBRARY_PATH to override this > as the env var will apply to both qemu-arm (an x86_64 binary) and the > binary it is trying to run (an arm binary). > > More typical though is that you have a directory containing an fullish > install tree of a non-native architecture and you just want to chroot > into that. When doing such a chroot, the qemu-$ARCH emulator must be > present inside the chroot too. ie the x86_64 build of /usr/bin/qemu-arm > must be present inside at /my/chroot/for/fedora-arm/usr/bin/qemu-arm. > So again you have the potential problem of clashing libc.so in /usr/lib > It is a shame Fedora doesn't have full multi-arch support, instead of > merely multi-lib to avoid these clashing lib dirs across architecture > RPMs. > > The recommended way to deal with this for the qemu user emulator binaries > to be statically linked, so when copied inside the non-native arch chroot, > they never need to resolve any native arch libraries. Fedora's qemu user > binaries are all dynamic linked right now. jfyi, nothing really to contribute on how to do this, but this has been a long-standing gripe of mine about fedora, and somehow packaging qemu-static is something I'm a really big fan of. (in a previous life, using sbox2 to "natively" build arm stuff on x86 was quite useful..) BR, -R > Debian handles this by having several packages [1] > > - qemu-user - the dynamic linked qemu user binaries > - qemu-binfmt - binfmt rules registering the dynamic linked binaries > - qemu-user-static - the static linked qemu user binaries *and* binfmt > rules to register them. The static binaries all > have -static suffix on their name > > NB, this means qemu-binfmt and qemu-user-static are mutually exclusive > since they both provide the same binfmt files. You can however have both > qemu-user and qemu-user-static installed as their binary names won't > clash, and in this case the static ones will be registered as binfmts > > This nice thing about this multiple package approach is that when you > copied the x86_64 build of the "qemu-arm-static" binary into your arm > chroot, you still then have the possibility of installing the arm build > of the "qemu-arm" binary inside that chroot without filename clash. > > An alternative simpler approach would be to just have one package, > qemu-user, which contains the static binaries and never ship any > dynamic linked qemu user binaries. This is slightly more restrictive > though, as explained in the previous paragraph, so I'd like to avoid > doing that. > > > I'd like to make using non-native arch chroots simple with Fedora without > people needing to manually build their own static QEMU binaries, or download > static binaries provided by another distro[2]. So I'm suggesting to make a > change to Fedora qemu packages to essentially copy the way Debian has done > things. Specifically I will > > - Pull the binfmt registration files out of qemu-user and into a > new qemu-binfmt package which depends on qemu-user. > > - Add static builds of qemu user emulators to a new qemu-user-static > package, along with binfmt registration files > > The static build of QEMU user emulators is moderately light on > dependancies, only requiring glib2-static, pcre-static, zlib-static > and glibc-static packages. > > The change to introduce a qemu-binfmt package has small upgrade > implications since anyone with qemu-user installed today, will loose > the binary format rules unless they manually install qemu-binfmt. I > think the number of people affected is probably quite small, and some > of them may well wish to use qemu-user-static instead anyway. > > Obviously this would only be done in rawhide, not any existing stable > releases of Fedora. > > Nothing will change about the rest of QEMU packaging - ie all system > emulators will continue to use dynamic linking > > Regards, > Daniel > > [1] https://wiki.debian.org/QemuUserEmulation > [2] https://rwmj.wordpress.com/2013/12/22/how-to-run-aarch64-binaries-on-an-x86-64-host-using-qemu-userspace-emulation/ > -- > |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| > |: http://libvirt.org -o- http://virt-manager.org :| > |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| > |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| > -- > devel mailing list > devel@xxxxxxxxxxxxxxxxxxxxxxx > https://lists.fedoraproject.org/admin/lists/devel@xxxxxxxxxxxxxxxxxxxxxxx -- devel mailing list devel@xxxxxxxxxxxxxxxxxxxxxxx https://lists.fedoraproject.org/admin/lists/devel@xxxxxxxxxxxxxxxxxxxxxxx
