On Wed, Jun 29, 2016 at 06:45:36AM -0700, Neal Gompa wrote: > On Wed, Jun 29, 2016 at 3:03 AM, Daniel P. Berrange <berrange@xxxxxxxxxx> wrote: > > For those who aren't familiar, QEMU actually provides two completely > > different sets of emulators > > > > - system emulators - they emulate a full virtual machine and thus run > > a full guest OS. > > - user emulators - they emulate the Linux userspace ABI letting you > > run non-native arch executables directly. > > > > The user emulators are what I'm concerned with in this mail, so ignore > > the system emulators. > > > > Currently all the user emulators are provided in the "qemu-user" RPM > > which also includes files in /usr/lib/binfmt.d to register each emulator > > binary as a binary format handler for its respective architecture. > > > > This is ok if you have a non-native arch binary that's statically linked > > and you just want to run it from context of your main OS root filesystem. > > Running dynamic linked binaries won't fly because if say running an arm > > binary on x86_64 host, it'll look for /lib/libc.so and find the i386 one, > > instead of the arm one. You can't set LD_LIBRARY_PATH to override this > > as the env var will apply to both qemu-arm (an x86_64 binary) and the > > binary it is trying to run (an arm binary). > > > > More typical though is that you have a directory containing an fullish > > install tree of a non-native architecture and you just want to chroot > > into that. When doing such a chroot, the qemu-$ARCH emulator must be > > present inside the chroot too. ie the x86_64 build of /usr/bin/qemu-arm > > must be present inside at /my/chroot/for/fedora-arm/usr/bin/qemu-arm. > > So again you have the potential problem of clashing libc.so in /usr/lib > > It is a shame Fedora doesn't have full multi-arch support, instead of > > merely multi-lib to avoid these clashing lib dirs across architecture > > RPMs. > > > > The recommended way to deal with this for the qemu user emulator binaries > > to be statically linked, so when copied inside the non-native arch chroot, > > they never need to resolve any native arch libraries. Fedora's qemu user > > binaries are all dynamic linked right now. > > > > Debian handles this by having several packages [1] > > > > - qemu-user - the dynamic linked qemu user binaries > > - qemu-binfmt - binfmt rules registering the dynamic linked binaries > > - qemu-user-static - the static linked qemu user binaries *and* binfmt > > rules to register them. The static binaries all > > have -static suffix on their name > > > > NB, this means qemu-binfmt and qemu-user-static are mutually exclusive > > since they both provide the same binfmt files. You can however have both > > qemu-user and qemu-user-static installed as their binary names won't > > clash, and in this case the static ones will be registered as binfmts > > > > This nice thing about this multiple package approach is that when you > > copied the x86_64 build of the "qemu-arm-static" binary into your arm > > chroot, you still then have the possibility of installing the arm build > > of the "qemu-arm" binary inside that chroot without filename clash. > > > > An alternative simpler approach would be to just have one package, > > qemu-user, which contains the static binaries and never ship any > > dynamic linked qemu user binaries. This is slightly more restrictive > > though, as explained in the previous paragraph, so I'd like to avoid > > doing that. > > > > > > I'd like to make using non-native arch chroots simple with Fedora without > > people needing to manually build their own static QEMU binaries, or download > > static binaries provided by another distro[2]. So I'm suggesting to make a > > change to Fedora qemu packages to essentially copy the way Debian has done > > things. Specifically I will > > > > - Pull the binfmt registration files out of qemu-user and into a > > new qemu-binfmt package which depends on qemu-user. > > > > - Add static builds of qemu user emulators to a new qemu-user-static > > package, along with binfmt registration files > > > > The static build of QEMU user emulators is moderately light on > > dependancies, only requiring glib2-static, pcre-static, zlib-static > > and glibc-static packages. > > > > The change to introduce a qemu-binfmt package has small upgrade > > implications since anyone with qemu-user installed today, will loose > > the binary format rules unless they manually install qemu-binfmt. I > > think the number of people affected is probably quite small, and some > > of them may well wish to use qemu-user-static instead anyway. > > > > Obviously this would only be done in rawhide, not any existing stable > > releases of Fedora. > > > > Nothing will change about the rest of QEMU packaging - ie all system > > emulators will continue to use dynamic linking > > > > Regards, > > Daniel > > > > > Amazingly, I've been pre-empted here. I was going to ask about this > for Fedora, since we've got interest in this for Mageia as part of > potentially enabling cross-arch building with mock in an easy fashion. > > Mageia tries to keep its QEMU package in sync with Fedora, so I'd love > to see this happen so that it becomes easier to maintain the > qemu-user-static stuff. > > Right now, we only have a couple of the arches built, but one of our > guys interested in this might be willing to help out on this side of > the fence with it (Per Øyvind Karlsen). I've CC'd him to this email. FYI, I have a test scratch-build with the proposed change: http://koji.fedoraproject.org/koji/taskinfo?taskID=14702004 This is completely untested by me so far, so may have bugs,m but it serves to illustrate the goal. Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| -- devel mailing list devel@xxxxxxxxxxxxxxxxxxxxxxx https://lists.fedoraproject.org/admin/lists/devel@xxxxxxxxxxxxxxxxxxxxxxx
