On Wed, 2016-06-29 at 15:02 +0100, Daniel P. Berrange wrote: > On Wed, Jun 29, 2016 at 09:39:11AM -0400, Simo Sorce wrote: > > On Wed, 2016-06-29 at 11:34 +0100, Daniel P. Berrange wrote: > > > On Wed, Jun 29, 2016 at 12:15:02PM +0200, Florian Weimer wrote: > > > > On 06/29/2016 12:03 PM, Daniel P. Berrange wrote: > > > > > Debian handles this by having several packages [1] > > > > > > > > > > - qemu-user - the dynamic linked qemu user binaries > > > > > - qemu-binfmt - binfmt rules registering the dynamic linked binaries > > > > > - qemu-user-static - the static linked qemu user binaries *and* binfmt > > > > > rules to register them. The static binaries all > > > > > have -static suffix on their name > > > > > > > > Debian builds static libraries and puts them into their -dev packages. > > > > Fedora does not do this systematically. Are all the static libraries you > > > > need actually available in Fedora? > > > > > > Yes, everything we need exists - as mentioned its only glibc-static, > > > glib2-static, zlib-static and pcre-static that we need for this. > > > > > > > An alternative would be to bind-mount the real file system into the chroot > > > > and run the native dynamic linker with a --library-path command line > > > > argument that specifies the library directories in the bind mount. (It's > > > > reasonable to specify --inhibit-cache as well.) This would need some > > > > adjustments in the binfmt wrapper. > > > > > > The binfmt registrations are global to the OS, so the same binfmt command > > > needs to work whether inside or outside the chroot. So a scheme which > > > requires us to pass special args to make the linker looks elsewhere when > > > inside the chroot is not so easy. We'd have to create a wrapper program > > > around the real qemu user binary that decides which args to pass to the > > > linker, and that wrapper would then have to be statically linked > > > > Just an idea: > > > > Would it make sense to build the qemu-user binary so that it looks in a > > different /lib directory (using rpath) like /lib/qemu-user-systemlib, > > and then symlink into that directory the libraries it needs from the > > host ? > > > > That way if you use chroot and bind mount in that directory in the right > > place the qemu-user binary uses different libraries from the emulated > > binaries yet you do not have to resort to static linking. > > I think you would need multiple levels of symlinking > > On the host root > > symlink: /usr/lib/qemu-user-root -> / > dir: /usr/lib/qemu-user-host-lib > symlink: /usr/lib/qemu-user-host-lib/libc.so -> /usr/lib/qemu-user-root/lib/libc.so > > And build qemu-$ARCH with rpath to /usr/lib/qemu-user-root/lib. > > Then in the chroot you would need to bind mount the host root into > /usr/lib/qemu-user-root, to override the symlink that would otherwise > point back to / Why do you need 2 symlinks ? I was thinking you'd have symlink: /usr/lib/qemu-user-lib-x86_64/ -> /usr/lib In the chroot you just bind mount the host's /usr/lib as /usr/lib/qemu-user-lib-x86_64 This assuming x86-64, you can bind mount in the appropriate arch-target dir on other arches. > Even if you do all that, you've now prevented installation of the > foreign arch's qemu-user RPM inside the chroot, See above, I think my solution would not prevent that. > as that will clash > with the one you've setup from the host. Also the way you deal with > cross-arch chroots is now different (and more complex) on Fedora, vs > every other Linux distro which just provides a static qemu-$ARCH you > can copy without needing to care about bind mounting extra dirs. Yes, bind-mount is an additional step, but doesn't look like a huge price to pay. Could even be done transparently by providing a qemu-user.sh script that "does the right thing" > So yes, it is probably possible, but it is not very appealing IMHO Thought it may be neat, but if consistency with other distros is what you are after then I would also lean to build a static binary. It's just that linking in glibc statically is ... not great. Simo. -- Simo Sorce * Red Hat, Inc * New York -- devel mailing list devel@xxxxxxxxxxxxxxxxxxxxxxx https://lists.fedoraproject.org/admin/lists/devel@xxxxxxxxxxxxxxxxxxxxxxx
