Re: [PATCH kvmtool 0/2] Add libfdt library

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi,

On Thu, May 26, 2022 at 04:59:07PM +0100, Andre Przywara wrote:
> On Thu, 26 May 2022 16:30:56 +0100
> Alexandru Elisei <alexandru.elisei@xxxxxxx> wrote:
> 
> Hi Alex,
> 
> > I found it very difficult to cross-build kvmtool for all arches because for
> > some architectures I couldn't get libfdt to install: either dtc wasn't
> > compiling (x86 machine, Arch Linux, missing yaml.h include when building
> > for arm and powerpc), or libfdt was not being found by Makefile after
> > installing it (x86 machine, Debian 11, building for powerpc). It was
> > possible to compile only libfdt and teach Makefile where to look for it,
> > which is what I ended up doing, but in my opinion, that's not an ideal user
> > experience.
> 
> Meh. Nobody said that userland cross compilation would be a walk in the
> park, and this approach (cross-compiling and putting in SYSROOT) is the
> canonical approach. When using a multiarch capable distro, this is easy
> enough.

Except when it doesn't work. On debian 11 I was not able to make it work for
powerpc by following the instructions, while I was able to make it work for
aarch64 following the same instructions. That's the kind of thing that I would
like to avoid.

> But I see that this is some major pain point for an otherwise rather "low
> maintenance" tool, and libfdt is designed to be embedded, so I guess this
> is fine, with the caveat mentioned below.
> 
> > This is my attempt at making compilation easier by pulling in the libfdt
> > library and having kvmtool compile it when an architecture requires it. All
> > the user needs to do now is to find a suitable cross-compiler toolchain
> > with glibc included [1]*.
> > 
> > When making lkvm-static, the static version of libfdt is built (libfdt.a);
> > for all other make targets the shared library is built.
> 
> So does that mean that the .so gets shipped with kvmtool? And it should be

Err.. I screwed up badly here, I just checked with ldd and lkvm was still
using the system proved libfdt.so, that's why it was still working.
Uninstalling the libfdt package broke lkvm.

> installed as part of "make install"? I think that might bring up more
> questions and problems, so do we want to leave the user some choice?
> - The user brings their own libfdt. This should be the case for any clean
> userland cross compilation, where you install target libraries in SYSROOT,
> just for compilation, and have the .so file also on the actual target
> system. This would also be best for native compilation. Current situation.
> - The user decides to use the built-in libfdt, in which case that gets
> always statically linked into the lkvm binary (which might still be
> dynamically linked against libc), to avoid any conflict between the just
> built version (.so), and the installed library on the target system.
> 
> Some projects choose this approach, where it gets decided at build time
> which version to use ("system library" vs. built-in version). This is
> either done automatically, because a system library is not found, or using
> some user provided flag for configure/make.
> 
> If that sounds too involved, we could just always link in our libfdt copy
> statically. And later revisit this when people complain.

I would prefer this approach, as it keeps things the Makefile simple and it
makes the build reproducible.

> Can you check by how much an otherwise dynamically linked kvmtool binary
> grows when we link against libfdt.a?

I wrote a patch to always link against libfdt.a, here are the results of
running make on a rockpro64 (I renamed the executables to tell which is
which).

Built from current master:

$ /usr/bin/ls -l lkvm-libfdt-dynamically-linked
-rwxr-xr-x 1 alex users 2127112 May 26 16:31 lkvm-libfdt-dynamically-linked
$ ldd lkvm-libfdt-dynamically-linked
	linux-vdso.so.1 (0x0000ffffb52ae000)
	libz.so.1 => /usr/lib/libz.so.1 (0x0000ffffb5140000)
	libaio.so.1 => /usr/lib/libaio.so.1 (0x0000ffffb5120000)
	libfdt.so.1 => /usr/lib/libfdt.so.1 (0x0000ffffb5100000)
	libc.so.6 => /usr/lib/libc.so.6 (0x0000ffffb4f50000)
	/lib/ld-linux-aarch64.so.1 => /usr/lib/ld-linux-aarch64.so.1 (0x0000ffffb5275000)

Built with Makefile modified to link against the static version:

$ /usr/bin/ls -l lkvm-libfdt-statically-linked
-rwxr-xr-x 1 alex users 2238920 May 26 16:32 lkvm-libfdt-statically-linked
$ ldd lkvm-libfdt-statically-linked
	linux-vdso.so.1 (0x0000ffff99a6d000)
	libz.so.1 => /usr/lib/libz.so.1 (0x0000ffff99900000)
	libaio.so.1 => /usr/lib/libaio.so.1 (0x0000ffff998e0000)
	libc.so.6 => /usr/lib/libc.so.6 (0x0000ffff99730000)
	/lib/ld-linux-aarch64.so.1 => /usr/lib/ld-linux-aarch64.so.1 (0x0000ffff99a34000)

A difference of 109KB (111808 bytes to be more exact).

I'll send the changes to the Makefile in case people want to play with it.

Thanks,
Alex

> 
> Cheers,
> Andre
> 
> > This only happens
> > for those arches that require libfdt (arm, arm64, riscv, powerpc); libfdt
> > is not built for MIPS and x86.
> > 
> > I am pretty happy with how libfdt has been integrated with the kvmtool
> > build system. The only niggle is that the libfdt_clean target echoes the
> > commands to remove the build artifacts:
> > 
> > $ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- clean
> > Makefile:378: Skipping optional libraries: bfd zlib aio
> >   CLEAN (libfdt)
> > rm -f libfdt/*.o libfdt/libfdt.a libfdt/libfdt-1.6.1.so
> > rm -f libfdt/libfdt.so.1
> >   CLEAN
> > 
> > but I believe that's impossible to change without modifying the libfdt
> > Makefile, which I don't want to do to keep it as easy as possible to update
> > libfdt.
> > 
> > Made some quick measurement about how libfdt affects the build time when
> > building from a clean directory (incremental builds do not rebuild libfdt):
> > 
> > - On a rockpro64, on the little core (Cortex-A53), running
> > 
> > $ time taskset -c 0 make
> > 
> > with libfdt takes an average of 90.721s (averaged over 3 runs, standard
> > deviation 0.040s); without libfdt it takes 84.003s (averaged over 3 runs,
> > stddev of 0.105s). That's a 6.718s absolute increase, or a 7.997% relative
> > increase.
> > 
> > - On the same rockpro64, on the big core (Cortex-A72), running
> > 
> > $ time taskset -c 5 make
> > 
> > with libfdt takes an average of 45.078s (averaged over 3 runs, stddev of
> > 0.060s); without libfdt it takes 41.836s (averaged over 3 runs, stddev of
> > 0.065). That's a 3.242s absolute increase, or a 7.749% relative increase.
> > 
> > I believe that the ease of use more than justifies the increased
> > compilation time.
> > 
> > Testing
> > =======
> > 
> > On x86 host, compiled for x86, MIPS, arm, arm64, powerpc, riscv. Booted an
> > x86 VM.
> > 
> > On arm64 host, compiled for arm64 and arm. I tried to cross-compile for
> > other targets, but I couldn't find a cross-compiler toolchain for arm64
> > hosts that included a libc implementation. The arm cross-compiler was
> > downloaded from the developer.arm.com [2]. Ran kvm-unit-tests for arm64 and
> > arm (on the arm64 version of kvmtool), and also booted a kernel.
> > 
> > *only for x86 hosts.
> > 
> > [1] https://toolchains.bootlin.com/
> > [2] https://developer.arm.com/tools-and-software/open-source-software/developer-tools/gnu-toolchain/downloads
> > 
> > Alexandru Elisei (2):
> >   Add libfdt library
> >   INSTALL: Remove all mentions of libfdt
> > 
> >  INSTALL                  |   27 +-
> >  Makefile                 |   53 +-
> >  libfdt/.gitignore        |    1 +
> >  libfdt/Makefile.libfdt   |   18 +
> >  libfdt/TODO              |    3 +
> >  libfdt/fdt.c             |  335 ++++++
> >  libfdt/fdt.h             |   66 ++
> >  libfdt/fdt_addresses.c   |  101 ++
> >  libfdt/fdt_check.c       |   93 ++
> >  libfdt/fdt_empty_tree.c  |   38 +
> >  libfdt/fdt_overlay.c     |  867 +++++++++++++++
> >  libfdt/fdt_ro.c          |  859 +++++++++++++++
> >  libfdt/fdt_rw.c          |  500 +++++++++
> >  libfdt/fdt_strerror.c    |   60 ++
> >  libfdt/fdt_sw.c          |  384 +++++++
> >  libfdt/fdt_wip.c         |   94 ++
> >  libfdt/libfdt.h          | 2147 ++++++++++++++++++++++++++++++++++++++
> >  libfdt/libfdt_env.h      |   96 ++
> >  libfdt/libfdt_internal.h |  192 ++++
> >  libfdt/meson.build       |   55 +
> >  libfdt/version.lds       |   83 ++
> >  21 files changed, 6033 insertions(+), 39 deletions(-)
> >  create mode 100644 libfdt/.gitignore
> >  create mode 100644 libfdt/Makefile.libfdt
> >  create mode 100644 libfdt/TODO
> >  create mode 100644 libfdt/fdt.c
> >  create mode 100644 libfdt/fdt.h
> >  create mode 100644 libfdt/fdt_addresses.c
> >  create mode 100644 libfdt/fdt_check.c
> >  create mode 100644 libfdt/fdt_empty_tree.c
> >  create mode 100644 libfdt/fdt_overlay.c
> >  create mode 100644 libfdt/fdt_ro.c
> >  create mode 100644 libfdt/fdt_rw.c
> >  create mode 100644 libfdt/fdt_strerror.c
> >  create mode 100644 libfdt/fdt_sw.c
> >  create mode 100644 libfdt/fdt_wip.c
> >  create mode 100644 libfdt/libfdt.h
> >  create mode 100644 libfdt/libfdt_env.h
> >  create mode 100644 libfdt/libfdt_internal.h
> >  create mode 100644 libfdt/meson.build
> >  create mode 100644 libfdt/version.lds
> > 
> 
_______________________________________________
kvmarm mailing list
kvmarm@xxxxxxxxxxxxxxxxxxxxx
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm



[Index of Archives]     [Linux KVM]     [Spice Development]     [Libvirt]     [Libvirt Users]     [Linux USB Devel]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]

  Powered by Linux