As normal user,instead,I don't see the previous errors,but virt-manager is missing :
marietto@chromarietto:~$ libvirtd &
[1] 1947
marietto@chromarietto:~$ ps ax | grep libvirt
1947 pts/0 Sl 0:00 libvirtd
1974 pts/0 S+ 0:00 grep libvirt
marietto@chromarietto:~$ virt-manager
-bash: virt-manager: command not found
On Fri, Aug 25, 2023 at 3:35 PM Mario Marietto <marietto2008@xxxxxxxxx> wrote:
Finally we can understand well what's wrong :root@chromarietto:/home/marietto/Desktop/Dati/qemu-v5.1.0# libvirtd &
[1] 1784root@chromarietto:/home/marietto/Desktop/Dati/qemu-v5.1.0# 2023-08-25 13:32:41.245+0000: 1801: info : libvirt version: 9.7.02023-08-25 13:32:41.245+0000: 1801: info : hostname: chromarietto2023-08-25 13:32:41.245+0000: 1801: error : virFirewallApplyRuleDirect:518 : internal error: Failed to apply firewall rules /usr/sbin/iptables -w --table mangle --list-rules: iptables v1.8.9 (legacy): can't initialize iptables table `mangle': Table does not exist (do you need to insmod?)Perhaps iptables or your kernel needs to be upgraded.
2023-08-25 13:32:41.263+0000: 1801: error : virFirewallApplyRuleDirect:518 : internal error: Failed to apply firewall rules /usr/sbin/ip6tables -w --table filter --list-rules: modprobe: FATAL: Module ip6_tables not found in directory /lib/modules/5.4.244-stb-cbe
ip6tables v1.8.9 (legacy): can't initialize ip6tables table `filter': Table does not exist (do you need to insmod?). Perhaps ip6tables or your kernel needs to be upgraded.
2023-08-25 13:32:41.693+0000: 1801: error : virGetUserID:760 : invalid argument: Failed to parse user 'libvirt-qemu'
2023-08-25 13:32:41.699+0000: 1801: error : virStateInitialize:672 : Initialization of QEMU state driver failed: invalid argument: Failed to parse user 'libvirt-qemu'
2023-08-25 13:32:41.699+0000: 1801: error : daemonRunStateInit:617 : Driver state initialization failedOn Fri, Aug 25, 2023 at 2:51 PM Mario Marietto <marietto2008@xxxxxxxxx> wrote:it does not care,I solve it following this small tutorial :thanks anyway. You are very gentle with me.On Fri, Aug 25, 2023 at 2:39 PM Mario Marietto <marietto2008@xxxxxxxxx> wrote:On Debian bookworm I get a compilation error,but not on Devuan :root@chromarietto:/home/marietto/Desktop/Dati/qemu-v5.1.0# ./configure --disable-werror --target-list=arm-softmmu --enable-opengl --enable-gtk --enable-kvm --enable-guest-agent --enable-spice --audio-drv-list="oss pa" --enable-libusb --enable-trace-backend=simple --enable-debug --prefix=/usr/localInstall prefix /usr/local
BIOS directory /usr/local/share/qemu
firmware path /usr/local/share/qemu-firmware
binary directory /usr/local/bin
library directory /usr/local/lib
module directory /usr/local/lib/qemu
libexec directory /usr/local/libexec
include directory /usr/local/include
config directory /usr/local/etc
local state directory /usr/local/var
Manual directory /usr/local/share/man
ELF interp prefix /usr/gnemul/qemu-%M
Build directory /home/marietto/Desktop/Dati/qemu-v5.1.0
Source path /home/marietto/Desktop/Dati/qemu-v5.1.0
GIT binary git
GIT submodules ui/keycodemapdb tests/fp/berkeley-testfloat-3 tests/fp/berkeley-softfloat-3 dtc capstone slirp
C compiler cc
Host C compiler cc
C++ compiler c++
Objective-C compiler cc
ARFLAGS rv
CFLAGS -g
QEMU_CFLAGS -I/usr/include/pixman-1 -I$(SRC_PATH)/dtc/libfdt -pthread -I/usr/include/glib-2.0 -I/usr/lib/arm-linux-gnueabihf/gl
ib-2.0/include -fPIE -DPIE -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wund
ef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv -std=gnu99 -Wold-style-declaration -Wold-style-defini
tion -Wtype-limits -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wempty-body -Wnested-externs -Wendif-labels -Wexpan
sion-to-defined -Wno-missing-include-dirs -Wno-shift-negative-value -Wno-psabi -fstack-protector-strong -I/usr/include/p11-kit-1 -D
STRUCT_IOVEC_DEFINED -I/usr/include/libpng16 -I/usr/include/spice-1 -I/usr/include/spice-server -I$(SRC_PATH)/capstone/include
QEMU_LDFLAGS -L$(BUILD_DIR)/dtc/libfdt -Wl,--warn-common -Wl,-z,relro -Wl,-z,now -pie -fstack-protector-strong
make make
install install
python /usr/bin/python3 -B (3.11.2)
genisoimage
efi_aarch64 /home/marietto/Desktop/Dati/qemu-v5.1.0/pc-bios/edk2-aarch64-code.fd
python_yaml no
slirp support git
smbd /usr/sbin/smbd
module support no
alt path mod load no
host CPU arm
host big endian no
target list arm-softmmu
gprof enabled no
sparse enabled no
strip binaries no
profiler no
static build no
safe stack no
SDL support no
SDL image support no
GTK support yes (3.24.37)
GTK GL support yes
VTE support no
TLS priority NORMAL
GNUTLS support yes
libgcrypt no
nettle yes (3.8.1)
XTS yes
libtasn1 yes
PAM no
iconv support yes
curses support yes
virgl support no
curl support yes
mingw32 support no
Audio drivers oss pa
Block whitelist (rw)
Block whitelist (ro)
VirtFS support yes
Multipath support no
VNC support yes
VNC SASL support yes
VNC JPEG support yes
VNC PNG support yes
xen support yes
xen ctrl version 41700
brlapi support no
Documentation no
PIE yes
vde support no
netmap support no
Linux AIO support no
Linux io_uring support no
ATTR/XATTR support yes
Install blobs yes
KVM support yes
HAX support no
HVF support no
WHPX support no
TCG support yes
TCG debug enabled yes
TCG interpreter no
malloc trim support yes
RDMA support no
PVRDMA support no
fdt support git
membarrier no
preadv support yes
fdatasync yes
madvise yes
posix_madvise yes
posix_memalign yes
libcap-ng support yes
vhost-net support yes
vhost-crypto support yes
vhost-scsi support yes
vhost-vsock support yes
vhost-user support yes
vhost-user-fs support yes
vhost-vdpa support yes
Trace backends simple
Trace output file trace-<pid>
spice support yes (0.14.3/0.15.1)
rbd support yes
xfsctl support no
smartcard support no
libusb yes
usb net redir no
OpenGL support yes
OpenGL dmabufs yes
libiscsi support yes
libnfs support no
build guest agent yes
QGA VSS support no
QGA w32 disk info no
QGA MSI support no
seccomp support no
coroutine backend ucontext
coroutine pool yes
debug stack usage no
mutex debugging yes
crypto afalg no
GlusterFS support yes
gcov gcov
gcov enabled no
TPM support yes
libssh support yes
QOM debugging yes
Live block migration yes
lzo support no
snappy support no
bzip2 support no
lzfse support no
zstd support yes
NUMA host support yes
libxml2 yes
tcmalloc support no
jemalloc support no
avx2 optimization
avx512f optimization no
replication support yes
bochs support yes
cloop support yes
dmg support yes
qcow v1 support yes
vdi support yes
vvfat support yes
qed support yes
parallels support yes
sheepdog support yes
capstone git
libpmem support no
libdaxctl support no
libudev yes
default devices yes
plugin support no
fuzzing support no
gdb
rng-none no
Linux keyring yes
cross containers no
NOTE: guest cross-compilers enabled: ccroot@chromarietto:/home/marietto/Desktop/Dati/qemu-v5.1.0# apt install libfdt-dev libdt-utils-dev python3-libfdtroot@chromarietto:/home/marietto/Desktop/Dati/qemu-v5.1.0# makeGEN arm-softmmu/config-devices.mak.tmp
GEN arm-softmmu/config-devices.mak
GEN config-all-devices.mak
GEN config-host.h
make[1]: Entering directory '/home/marietto/Desktop/Dati/qemu-v5.1.0/slirp'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/home/marietto/Desktop/Dati/qemu-v5.1.0/slirp'
make[1]: *** No rule to make target '/root/Desktop/qemu-v5.1.0/dtc/libfdt/fdt.h', needed by 'libfdt/fdt.o'. Stop.make: *** [Makefile:536: dtc/all] Error 2root@chromarietto:/home/marietto/Desktop/Dati/qemu-v5.1.0# cd dtcroot@chromarietto:/home/marietto/Desktop/Dati/qemu-v5.1.0/dtc# cd libfdtroot@chromarietto:/home/marietto/Desktop/Dati/qemu-v5.1.0/dtc/libfdt# lsfdt_addresses.c fdt_check.d fdt_empty_tree.o fdt_overlay.o fdt_rw.d fdt_sw.c fdt_wip.o Makefile.libfdt
fdt_addresses.d fdt_check.o fdt.h fdt_ro.c fdt_rw.o fdt_sw.d libfdt.a TODO
fdt_addresses.o fdt.d fdt.o fdt_ro.d fdt_strerror.c fdt_sw.o libfdt_env.h version.lds
fdt.c fdt_empty_tree.c fdt_overlay.c fdt_ro.o fdt_strerror.d fdt_wip.c libfdt.h
fdt_check.c fdt_empty_tree.d fdt_overlay.d fdt_rw.c fdt_strerror.o fdt_wip.d libfdt_internal.h
On Fri, Aug 25, 2023 at 10:51 AM Pavel Hrdina <phrdina@xxxxxxxxxx> wrote:On Thu, Aug 24, 2023 at 10:40:40PM +0200, Mario Marietto wrote:
> I've installed Debian bookworm instead of devuan. Maybe it's easier,since I
> know debian better than devuan. Let me understand a point : should I issue
> "apt build-dep libvirt" instead of "apt install virt-manager" ? aren't they
> equivalent commands ? Usually,as soon as I reinstall Debian,I do the
> installation of virt-manager and it grabs all its dependencies. If I
> understand correctly,I should not install virt-manager and its
> dependencies,right ? thanks.
If you run `apt install virt-manager` it will install virt-manager,
libvirt, qemu and everything needed to run and use the software.
If you run `apt build-dep libvirt` it will install everything needed to
compile libvirt but not necessarily everything you need to actually run
it. If you do only build-dep for libvirt you will have no virt-manager
and you would not be able to use it. You would have only libvirt.
> On Thu, Aug 24, 2023 at 3:16 PM Pavel Hrdina <phrdina@xxxxxxxxxx> wrote:
>
> > On Wed, Aug 23, 2023 at 08:47:55PM +0200, Mario Marietto wrote:
> > > I've fixed this problem :
> > >
> > > virsh: /lib/arm-linux-gnueabihf/libvirt.so.0: version
> > > `LIBVIRT_PRIVATE_9.0.0' not found (required by virsh)
> > >
> > > now,virsh works if I launch it within the build directory of libvirt :
> > >
> > > root@devuan:~/libvirt_build/bin# ./virsh
> > > Welcome to virsh, the virtualization interactive terminal.
> > >
> > > Type: 'help' for help with commands
> > > 'quit' to quit
> > >
> > > but inside the build directory of libvirt I don't see any virt-manager
> > > executable,so I'm forced to run it like this :
> > >
> > > root@devuan:~/Desktop/libvirt/build# ./run /usr/bin/virt-manager
> > >
> > > and it gives the previous error : no connection driver available for
> > > qemu:///system
> > >
> > > but the fact that virsh works is a progress ? should I compile
> > virt-manager
> > > also from source code ?
> >
> > No, there is no need to compile virt-manager, it is written in python
> > and so there is no compilation involved at all.
> >
> >
> > I would probably recommend you to remove `~/libvirt_build/` and
> > `~/Desktop/libvirt/build/` directories and start again.
> >
> > If you've successfully installed all libvirt build dependencies using
> > `apt build-dep libvirt` you should have everything to build your own
> > libvirt if you are on Devuan 5.
> >
> > The next step would be configuring libvirt so you can compile it using
> > the following command executed in the libvirt source directory:
> >
> > meson setup build
> >
> > Once that is completed you can build and install your libvirt using
> > the following command directly from the source directory as well:
> >
> > ninja -C build
> > sudo ninja -C build install
> >
> > The first one will build libvirt (I would not run it as root) and the
> > second one will install it into /usr/local/.
> >
> > After that is done you should be able to run virsh and libvirtd directly
> > without any run script. You can verify which binary is actually used if
> > you run the following:
> >
> > which virsh
> > which libvirtd
> >
> > If everything is correct you should get these paths:
> >
> > /usr/local/bin/virsh
> > /usr/local/sbin/libvirtd
> >
> > Now you should be able to simply run `libvirtd &` to start the daemon.
> >
> > After that running `virsh version` or any virsh command should just
> > work. Running `virt-manager` should also work and connect to the correct
> > libvirt.
> >
> >
> > I also tried installing QEMU build dependencies using `apt build-dep qemu`
> > and used the following steps to build QEMU 5.1.0 from source directory:
> >
> > mkdir build
> > cd build
> > ../configure (with options, I'm doing it on x86_64 so they are bit
> > different)
> > make
> > sudo make install
> >
> > But you should have theses steps completed already. Again you can verify
> > that you are using correct qemu-system-arm running:
> >
> > which qemu-system-arm
> >
> > where you should get
> >
> > /usr/local/bin/qemu-system-arm
> >
> > If you complete all of the build and install steps like here and start
> > libvirtd after that you should have everything set to work properly
> > and starting `virt-manager` without any special path should just work.
> >
> > I verified all of this on Devuan 5 but on x86_64 architecture, don't
> > have any 32bit arm to try it on.
> >
> > Pavel
> >
> >
> > > On Wed, Aug 23, 2023 at 7:39 PM Mario Marietto <marietto2008@xxxxxxxxx>
> > > wrote:
> > >
> > > > Despite that error,the previous error is still there :
> > > >
> > > > Unable to connect to libvirt qemu:///system.
> > > >
> > > > no connection driver available for qemu:///system
> > > >
> > > > Libvirt URI is: qemu:///system
> > > >
> > > > Traceback (most recent call last):
> > > > File "/usr/share/virt-manager/virtManager/connection.py", line 923,
> > in
> > > > _do_open
> > > > self._backend.open(cb, data)
> > > > File "/usr/share/virt-manager/virtinst/connection.py", line 171, in
> > open
> > > > conn = libvirt.openAuth(self._open_uri,
> > > > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> > > > File "/usr/lib/python3/dist-packages/libvirt.py", line 147, in
> > openAuth
> > > > raise libvirtError('virConnectOpenAuth() failed')
> > > > libvirt.libvirtError: no connection driver available for qemu:///system
> > > >
> > > > I'm using qemu 5.1 :
> > > >
> > > > root@devuan:~# qemu-system-arm --version
> > > >
> > > > QEMU emulator version 5.1.0 (v5.1.0-dirty)
> > > >
> > > >
> > > > On Wed, Aug 23, 2023 at 7:30 PM Mario Marietto <marietto2008@xxxxxxxxx
> > >
> > > > wrote:
> > > >
> > > >> root@devuan:~/Desktop/libvirt/build# sudo virsh list --all
> > > >>
> > > >> virsh: /lib/arm-linux-gnueabihf/libvirt.so.0: version
> > > >> `LIBVIRT_PRIVATE_9.0.0' not found (required by virsh)
> > > >>
> > > >>
> > > >>
> > > >> On Wed, Aug 23, 2023 at 5:43 PM Pavel Hrdina <phrdina@xxxxxxxxxx>
> > wrote:
> > > >>
> > > >>> On Wed, Aug 23, 2023 at 03:41:42PM +0200, Mario Marietto wrote:
> > > >>> > Hello Pavel,
> > > >>> >
> > > >>> > It didn't work,but the error was changed. Isn't a bad news :
> > > >>> >
> > > >>> > root@devuan:~/Desktop/libvirt/build# ./run /usr/bin/virt-manager
> > > >>> >
> > > >>> > Unable to connect to libvirt qemu:///system.
> > > >>> >
> > > >>> > no connection driver available for qemu:///system
> > > >>> >
> > > >>> > Libvirt URI is: qemu:///system
> > > >>> >
> > > >>> > Traceback (most recent call last):
> > > >>> > File "/usr/share/virt-manager/virtManager/connection.py", line
> > 923,
> > > >>> in
> > > >>> > _do_open
> > > >>> > self._backend.open(cb, data)
> > > >>> > File "/usr/share/virt-manager/virtinst/connection.py", line 171,
> > in
> > > >>> open
> > > >>> > conn = libvirt.openAuth(self._open_uri,
> > > >>> > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> > > >>> > File "/usr/lib/python3/dist-packages/libvirt.py", line 147, in
> > > >>> openAuth
> > > >>> > raise libvirtError('virConnectOpenAuth() failed')
> > > >>> > libvirt.libvirtError: no connection driver available for
> > qemu:///system
> > > >>>
> > > >>> Seems like you've compiled libvirt without qemu support. My guess is
> > > >>> that you are missing dependencies to compile libvirt with QEMU
> > driver.
> > > >>> If you run
> > > >>>
> > > >>> apt build-dep libvirt
> > > >>>
> > > >>> it should install you dependencies that libvirt from package manager
> > > >>> would need and that is a good starting point to compile your own
> > > >>> libvirt.
> > > >>>
> > > >>>
> > > >>> > On Wed, Aug 23, 2023 at 2:27 PM Pavel Hrdina <phrdina@xxxxxxxxxx>
> > > >>> wrote:
> > > >>> >
> > > >>> > > On Tue, Aug 22, 2023 at 10:23:32PM +0200, Mario Marietto wrote:
> > > >>> > > > After having compiled libvirt from src and having run
> > > >>> virt-manager,I get
> > > >>> > > > the error "Libvirtd daemon is not running". Can you give a
> > look at
> > > >>> this
> > > >>> > > > picture ?
> > > >>> > > >
> > > >>> > > > https://ibb.co/n6MvDbL
> > > >>> > > >
> > > >>> > > > how to fix it ? thanks.
> > > >>> > >
> > > >>> > > If you managed to start libvirtd using the run script from
> > libvirt
> > > >>> build
> > > >>> > > directory you should use the run script to start virt-manager as
> > > >>> well.
> > > >>> > >
> > > >>> > > > On Tue, Aug 22, 2023 at 10:17 PM Mario Marietto <
> > > >>> marietto2008@xxxxxxxxx>
> > > >>> > > > wrote:
> > > >>> > > >
> > > >>> > > > > [image: Screenshot from 2023-08-22 22-06-30.png]
> > > >>> > > > >
> > > >>> > > > > On Tue, Aug 22, 2023 at 9:44 PM Mario Marietto <
> > > >>> marietto2008@xxxxxxxxx
> > > >>> > > >
> > > >>> > > > > wrote:
> > > >>> > > > >
> > > >>> > > > >>
> > > >>> > > > >> Hello,
> > > >>> > > > >>
> > > >>> > > > >> Pavel,are you there ? Please help me to give it the last
> > push.
> > > >>> I've
> > > >>> > > fixed
> > > >>> > > > >> the last error like so :
> > > >>> > > > >>
> > > >>> > > > >> root@devuan:~/Desktop/libvirt/build# update-alternatives
> > --set
> > > >>> > > iptables
> > > >>> > > > >> /usr/sbin/iptables-legacy
> > > >>> > > > >> update-alternatives: using /usr/sbin/iptables-legacy to
> > provide
> > > >>> > > > >> /usr/sbin/iptables (iptables) in manual mode
> > > >>> > > > >>
> > > >>> > > > >> but now I have got another error. Anyway,I feel to be close
> > :
> > > >>> > > > >>
> > > >>> > > > >> root@devuan:~/Desktop/libvirt/build# sudo ./run
> > src/virtlockd
> > > >>> & sudo
> > > >>> > > > >> ./run src/virtlogd & sudo ./run src/libvirtd
> > > >>> > > > >>
> > > >>> > > > >> 2023-08-22 19:38:49.036+0000: 15002: info : libvirt version:
> > > >>> 9.7.0
> > > >>> > > > >>
> > > >>> > > > >> hostname: devuan
> > > >>> > > > >> error : virPidFileAcquirePathFull:409 : Failed to acquire
> > pid
> > > >>> file
> > > >>> > > > >> '/root/libvirt_build/var/run/virtlockd.pid': Resource
> > > >>> temporarily
> > > >>> > > > >> unavailable
> > > >>> > > > >>
> > > >>> > > > >> On Tue, Aug 22, 2023 at 7:01 PM Mario Marietto <
> > > >>> > > marietto2008@xxxxxxxxx>
> > > >>> > > > >> wrote:
> > > >>> > > > >>
> > > >>> > > > >>> Et voila'. I've recompiled libvirt in this way :
> > > >>> > > > >>>
> > > >>> > > > >>> git clone https://github.com/libvirt/libvirt.git
> > > >>> > > > >>> mkdir -p libvirt_build
> > > >>> > > > >>> cd libvirt
> > > >>> > > > >>> apt install meson xsltproc libgnutls28-dev libxml2-dev
> > rst2pdf
> > > >>> > > > >>> meson build --prefix=$HOME/libvirt_build
> > > >>> > > > >>> ninja -C build
> > > >>> > > > >>> sudo ninja -C build install
> > > >>> > > > >>> cd build
> > > >>> > > > >>> nano vai.sh :
> > > >>> > > > >>>
> > > >>> > > > >>> sudo ./run src/virtlockd & sudo ./run src/virtlogd & sudo
> > ./run
> > > >>> > > > >>> src/libvirtd
> > > >>> > > > >>>
> > > >>> > > > >>> chmod +x vai.sh
> > > >>> > > > >>>
> > > >>> > > > >>> this is the error I get :
> > > >>> > > > >>>
> > > >>> > > > >>> root@devuan:~/Desktop/libvirt/build# ./vai.sh
> > > >>> > > > >>>
> > > >>> > > > >>> 2023-08-22 16:52:34.416+0000: 14725: info : libvirt
> > version:
> > > >>> 9.7.0
> > > >>> > > > >>> 2023-08-22 16:52:34.416+0000: 14725: info : hostname:
> > devuan
> > > >>> > > > >>> 2023-08-22 16:52:34.416+0000: 14725: error :
> > > >>> > > > >>> virFirewallApplyRuleDirect:518 : internal error: Failed to
> > > >>> apply
> > > >>> > > > >>> firewall rules /usr/sbin/iptables -w --table filter
> > > >>> --list-rules:
> > > >>> > > > >>> iptables: Failed to initialize nft: Protocol not supported
> > > >>> > > > >>>
> > > >>> > > > >>> 2023-08-22 16:52:34.428+0000: 14725: error :
> > > >>> > > > >>> virFirewallApplyRuleDirect:518 : internal error: Failed to
> > > >>> apply
> > > >>> > > firewall
> > > >>> > > > >>> rules /usr/sbin/ip6tables -w --table filter --list-rules:
> > > >>> ip6tables:
> > > >>> > > Failed
> > > >>> > > > >>> to initialize nft: Protocol not supported
> > > >>> > > > >>>
> > > >>> > > > >>> On Tue, Aug 22, 2023 at 6:49 PM Pavel Hrdina <
> > > >>> phrdina@xxxxxxxxxx>
> > > >>> > > wrote:
> > > >>> > > > >>>
> > > >>> > > > >>>> On Tue, Aug 22, 2023 at 06:22:25PM +0200, Mario Marietto
> > > >>> wrote:
> > > >>> > > > >>>> > In your opinion should I remove virt-manager and all its
> > > >>> > > dependencies
> > > >>> > > > >>>> > before launching the recompilation ?
> > > >>> > > > >>>>
> > > >>> > > > >>>> I would say there is no need to do that if you install
> > > >>> libvirt to
> > > >>> > > > >>>> /usr/local as it should be separated from the libvirt
> > > >>> installed by
> > > >>> > > > >>>> package manager.
> > > >>> > > > >>>>
> > > >>> > > > >>>> Pavel
> > > >>> > > > >>>>
> > > >>> > > > >>>> >
> > > >>> > > > >>>> > On Tue, Aug 22, 2023 at 6:05 PM Pavel Hrdina <
> > > >>> phrdina@xxxxxxxxxx>
> > > >>> > > > >>>> wrote:
> > > >>> > > > >>>> >
> > > >>> > > > >>>> > > On Tue, Aug 22, 2023 at 05:55:30PM +0200, Mario
> > Marietto
> > > >>> wrote:
> > > >>> > > > >>>> > > > I have installed virt-manager and all its
> > dependencies
> > > >>> > > grabbing
> > > >>> > > > >>>> the files
> > > >>> > > > >>>> > > > from the devuan 5 repository.
> > > >>> > > > >>>> > > > To recompile libvirt I need to acquire some basic
> > > >>> information.
> > > >>> > > > >>>> Which
> > > >>> > > > >>>> > > > version should I compile ? Where is the source code
> > > >>> located ?
> > > >>> > > Do
> > > >>> > > > >>>> you have
> > > >>> > > > >>>> > > > some tutorial that explains the necessary steps to
> > do
> > > >>> it ?
> > > >>> > > thanks.
> > > >>> > > > >>>> > >
> > > >>> > > > >>>> > > Everything can be found at https://libvirt.org/ not
> > sure
> > > >>> which
> > > >>> > > > >>>> version
> > > >>> > > > >>>> > > to pick, you can try latest upstream version but there
> > > >>> might be
> > > >>> > > some
> > > >>> > > > >>>> > > dependencies missing in Devuan, if that happens you
> > can
> > > >>> > > fallback to
> > > >>> > > > >>>> > > version that is provided by Devuan.
> > > >>> > > > >>>> > >
> > > >>> > > > >>>> > > For example this link has basic info on how to compile
> > > >>> libvirt
> > > >>> > > > >>>> > > https://libvirt.org/compiling.html and here you can
> > find
> > > >>> > > sources
> > > >>> > > > >>>> > > to libvirt https://libvirt.org/downloads.html .
> > > >>> > > > >>>> > >
> > > >>> > > > >>>> > > Pavel
> > > >>> > > > >>>> > >
> > > >>> > > > >>>> > > > On Tue, Aug 22, 2023 at 5:50 PM Pavel Hrdina <
> > > >>> > > phrdina@xxxxxxxxxx>
> > > >>> > > > >>>> wrote:
> > > >>> > > > >>>> > > >
> > > >>> > > > >>>> > > > > On Tue, Aug 22, 2023 at 05:28:50PM +0200, Mario
> > > >>> Marietto
> > > >>> > > wrote:
> > > >>> > > > >>>> > > > > > root@devuan:~# virsh domcapabilities --machine
> > virt
> > > >>> > > > >>>> --emulatorbin
> > > >>> > > > >>>> > > > > > /usr/bin/qemu-system-arm
> > > >>> > > > >>>> > > > > >
> > > >>> > > > >>>> > > > > > error: failed to get emulator capabilities
> > > >>> > > > >>>> > > > > > error: KVM is not supported on this platform:
> > > >>> Function not
> > > >>> > > > >>>> > > implemented
> > > >>> > > > >>>> > > > > >
> > > >>> > > > >>>> > > > > > BUT it's not true :
> > > >>> > > > >>>> > > > > >
> > > >>> > > > >>>> > > > > > root@devuan:~# kvm-ok
> > > >>> > > > >>>> > > > > >
> > > >>> > > > >>>> > > > > > INFO: /dev/kvm exists
> > > >>> > > > >>>> > > > > > KVM acceleration can be used
> > > >>> > > > >>>> > > > >
> > > >>> > > > >>>> > > > > Are you using libvirt installed using package
> > manager
> > > >>> or
> > > >>> > > some
> > > >>> > > > >>>> libvirt
> > > >>> > > > >>>> > > > > that you compiled yorself? Checking libvirt code
> > this
> > > >>> error
> > > >>> > > only
> > > >>> > > > >>>> > > happens
> > > >>> > > > >>>> > > > > if libvirt was compiled without KVM, more
> > > >>> specifically if
> > > >>> > > header
> > > >>> > > > >>>> > > > > `/usr/include/linux/kvm.h` doesn't exists.
> > > >>> > > > >>>> > > > >
> > > >>> > > > >>>> > > > > I was not able to find any details on Devuan
> > distro
> > > >>> so not
> > > >>> > > sure
> > > >>> > > > >>>> how
> > > >>> > > > >>>> > > > > the libvirt package is compiled. You might need
> > to try
> > > >>> > > compiling
> > > >>> > > > >>>> > > libvirt
> > > >>> > > > >>>> > > > > yourself.
> > > >>> > > > >>>> > > > >
> > > >>> > > > >>>> > > > > Pavel
> > > >>> > > > >>>> > > > >
> > > >>> > > > >>>> > > > > >
> > > >>> > > > >>>> > > > > > On Tue, Aug 22, 2023 at 5:27 PM Mario Marietto <
> > > >>> > > > >>>> > > marietto2008@xxxxxxxxx>
> > > >>> > > > >>>> > > > > > wrote:
> > > >>> > > > >>>> > > > > >
> > > >>> > > > >>>> > > > > > > Oh sorry....
> > > >>> > > > >>>> > > > > > >
> > > >>> > > > >>>> > > > > > > On Tue, Aug 22, 2023 at 5:26 PM Mario
> > Marietto <
> > > >>> > > > >>>> > > marietto2008@xxxxxxxxx
> > > >>> > > > >>>> > > > > >
> > > >>> > > > >>>> > > > > > > wrote:
> > > >>> > > > >>>> > > > > > >
> > > >>> > > > >>>> > > > > > >> virsh domcapabilities --machine virt
> > > >>> --emulatorbin
> > > >>> > > > >>>> > > > > > >> /path/to/qemu-system-arm
> > > >>> > > > >>>> > > > > > >>
> > > >>> > > > >>>> > > > > > >> error: failed to get emulator capabilities
> > > >>> > > > >>>> > > > > > >> error: Cannot check QEMU binary
> > > >>> > > /path/to/qemu-system-arm:
> > > >>> > > > >>>> No such
> > > >>> > > > >>>> > > > > file or
> > > >>> > > > >>>> > > > > > >> directory
> > > >>> > > > >>>> > > > > > >>
> > > >>> > > > >>>> > > > > > >> On Tue, Aug 22, 2023 at 4:49 PM Pavel Hrdina
> > <
> > > >>> > > > >>>> phrdina@xxxxxxxxxx>
> > > >>> > > > >>>> > > > > wrote:
> > > >>> > > > >>>> > > > > > >>
> > > >>> > > > >>>> > > > > > >>> On Tue, Aug 22, 2023 at 04:05:09PM +0200,
> > Mario
> > > >>> > > Marietto
> > > >>> > > > >>>> wrote:
> > > >>> > > > >>>> > > > > > >>> > Where does libvirt want to find those
> > files ?
> > > >>> since
> > > >>> > > the
> > > >>> > > > >>>> qemu
> > > >>> > > > >>>> > > 5.1
> > > >>> > > > >>>> > > > > > >>> > installation files have been placed under
> > > >>> /usr/local
> > > >>> > > > >>>> during the
> > > >>> > > > >>>> > > > > command
> > > >>> > > > >>>> > > > > > >>> > make install,I have also copied the
> > firmware
> > > >>> files
> > > >>> > > in :
> > > >>> > > > >>>> > > > > > >>> >
> > > >>> > > > >>>> > > > > > >>> > ls /usr/local/share/AAVMF
> > > >>> > > > >>>> > > > > > >>> >
> > > >>> > > > >>>> > > > > > >>> > AAVMF32_CODE.fd AAVMF_CODE.fd
> > > >>> > > > >>>> AAVMF_CODE.snakeoil.fd
> > > >>> > > > >>>> > > > > > >>> AAVMF_VARS.ms.fd
> > > >>> > > > >>>> > > > > > >>> > AAVMF32_VARS.fd AAVMF_CODE.ms.fd
> > > >>> AAVMF_VARS.fd
> > > >>> > > > >>>> > > > > > >>> > AAVMF_VARS.snakeoil.fd
> > > >>> > > > >>>> > > > > > >>> >
> > > >>> > > > >>>> > > > > > >>> > but they aren't still recognized.
> > > >>> > > > >>>> > > > > > >>>
> > > >>> > > > >>>> > > > > > >>> Downgrading libvirt would not help in this
> > > >>> specific
> > > >>> > > case.
> > > >>> > > > >>>> Since
> > > >>> > > > >>>> > > > > version
> > > >>> > > > >>>> > > > > > >>> 5.2.0 libvirt uses firmware auto-selection.
> > > >>> Libvirt is
> > > >>> > > > >>>> looking
> > > >>> > > > >>>> > > for
> > > >>> > > > >>>> > > > > json
> > > >>> > > > >>>> > > > > > >>> files describing available firmwares. It is
> > > >>> > > documented in
> > > >>> > > > >>>> QEMU
> > > >>> > > > >>>> > > > > project
> > > >>> > > > >>>> > > > > > >>> git repository in
> > `docs/interop/firmware.json`,
> > > >>> this
> > > >>> > > > >>>> specific
> > > >>> > > > >>>> > > section
> > > >>> > > > >>>> > > > > > >>> describes where the json files should be
> > placed:
> > > >>> > > > >>>> > > > > > >>>
> > > >>> > > > >>>> > > > > > >>> # It is recommended to create firmware JSON
> > > >>> files
> > > >>> > > (each
> > > >>> > > > >>>> > > containing a
> > > >>> > > > >>>> > > > > > >>> # single @Firmware root element) with a
> > > >>> double-digit
> > > >>> > > > >>>> prefix, for
> > > >>> > > > >>>> > > > > example
> > > >>> > > > >>>> > > > > > >>> # "50-ovmf.json", "50-seabios-256k.json",
> > etc,
> > > >>> so they
> > > >>> > > > >>>> can be
> > > >>> > > > >>>> > > sorted
> > > >>> > > > >>>> > > > > in
> > > >>> > > > >>>> > > > > > >>> # predictable order. The firmware JSON files
> > > >>> should be
> > > >>> > > > >>>> searched
> > > >>> > > > >>>> > > for
> > > >>> > > > >>>> > > > > in
> > > >>> > > > >>>> > > > > > >>> # three directories:
> > > >>> > > > >>>> > > > > > >>> #
> > > >>> > > > >>>> > > > > > >>> # - /usr/share/qemu/firmware -- populated
> > by
> > > >>> > > > >>>> distro-provided
> > > >>> > > > >>>> > > > > firmware
> > > >>> > > > >>>> > > > > > >>> # packages
> > > >>> > > (XDG_DATA_DIRS
> > > >>> > > > >>>> covers
> > > >>> > > > >>>> > > > > > >>> #
> > /usr/share by
> > > >>> > > default),
> > > >>> > > > >>>> > > > > > >>> #
> > > >>> > > > >>>> > > > > > >>> # - /etc/qemu/firmware -- exclusively for
> > > >>> sysadmins'
> > > >>> > > > >>>> local
> > > >>> > > > >>>> > > > > additions,
> > > >>> > > > >>>> > > > > > >>> #
> > > >>> > > > >>>> > > > > > >>> # - $XDG_CONFIG_HOME/qemu/firmware --
> > > >>> exclusively
> > > >>> > > for
> > > >>> > > > >>>> per-user
> > > >>> > > > >>>> > > > > local
> > > >>> > > > >>>> > > > > > >>> #
> > > >>> additions
> > > >>> > > > >>>> > > (XDG_CONFIG_HOME
> > > >>> > > > >>>> > > > > > >>> #
> > > >>> defaults to
> > > >>> > > > >>>> > > $HOME/.config).
> > > >>> > > > >>>> > > > > > >>>
> > > >>> > > > >>>> > > > > > >>> It doesn't matter where the *CODE* and
> > *VARS*
> > > >>> firmware
> > > >>> > > > >>>> files are
> > > >>> > > > >>>> > > > > placed
> > > >>> > > > >>>> > > > > > >>> if the path to these files is correct in the
> > > >>> json
> > > >>> > > files
> > > >>> > > > >>>> in one
> > > >>> > > > >>>> > > of the
> > > >>> > > > >>>> > > > > > >>> three directories.
> > > >>> > > > >>>> > > > > > >>>
> > > >>> > > > >>>> > > > > > >>> Looking at the qemu-efi-arm package it
> > should
> > > >>> install
> > > >>> > > > >>>> > > > > > >>>
> > > >>> > > > >>>> > > > > > >>> /usr/share/AAVMF/AAVMF32_CODE.fd
> > > >>> > > > >>>> > > > > > >>> /usr/share/AAVMF/AAVMF32_VARS.fd
> > > >>> > > > >>>> > > > > > >>>
> > /usr/share/qemu/firmware/60-edk2-arm.json
> > > >>> > > > >>>> > > > > > >>>
> > > >>> > > > >>>> > > > > > >>> and that should be picked up correctly by
> > > >>> libvirt.
> > > >>> > > > >>>> > > > > > >>>
> > > >>> > > > >>>> > > > > > >>>
> > > >>> > > > >>>> > > > > > >>> I don't know what machine types are
> > available
> > > >>> for
> > > >>> > > 32bit
> > > >>> > > > >>>> ARM, but
> > > >>> > > > >>>> > > you
> > > >>> > > > >>>> > > > > > >>> should be able to figure that out by
> > running:
> > > >>> > > > >>>> > > > > > >>>
> > > >>> > > > >>>> > > > > > >>> virsh capabilities | grep canonical
> > > >>> > > > >>>> > > > > > >>>
> > > >>> > > > >>>> > > > > > >>> it will show only lines with machine types,
> > but
> > > >>> my
> > > >>> > > guess
> > > >>> > > > >>>> is on
> > > >>> > > > >>>> > > arm
> > > >>> > > > >>>> > > > > there
> > > >>> > > > >>>> > > > > > >>> should be 'virt' machine type so running
> > > >>> > > > >>>> > > > > > >>>
> > > >>> > > > >>>> > > > > > >>> virsh domcapabilities --machine virt
> > > >>> --emulatorbin
> > > >>> > > > >>>> > > > > > >>> /path/to/qemu-system-arm
> > > >>> > > > >>>> > > > > > >>>
> > > >>> > > > >>>> > > > > > >>> where you should be able to see the firmware
> > > >>> paths if
> > > >>> > > > >>>> they are
> > > >>> > > > >>>> > > > > correctly
> > > >>> > > > >>>> > > > > > >>> detected by libvirt.
> > > >>> > > > >>>> > > > > > >>>
> > > >>> > > > >>>> > > > > > >>> Pavel
> > > >>> > > > >>>> > > > > > >>>
> > > >>> > > > >>>> > > > > > >>>
> > > >>> > > > >>>> > > > > > >>> > On Tue, Aug 22, 2023 at 3:55 PM Mario
> > > >>> Marietto <
> > > >>> > > > >>>> > > > > marietto2008@xxxxxxxxx
> > > >>> > > > >>>> > > > > > >>> >
> > > >>> > > > >>>> > > > > > >>> > wrote:
> > > >>> > > > >>>> > > > > > >>> >
> > > >>> > > > >>>> > > > > > >>> > > I've already did that :
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > # apt install qemu-efi-arm
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > Reading package lists... Done
> > > >>> > > > >>>> > > > > > >>> > > Building dependency tree... Done
> > > >>> > > > >>>> > > > > > >>> > > Reading state information... Done
> > > >>> > > > >>>> > > > > > >>> > > qemu-efi-arm is already the newest
> > version
> > > >>> > > > >>>> (2022.11-6).
> > > >>> > > > >>>> > > > > > >>> > > qemu-efi-arm set to manually installed.
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > if I don't get wrong,that package do the
> > > >>> > > installation
> > > >>> > > > >>>> of the
> > > >>> > > > >>>> > > > > > >>> following
> > > >>> > > > >>>> > > > > > >>> > > files :
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > root@devuan:/usr/share/AAVMF# ls
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > AAVMF32_CODE.fd AAVMF_CODE.fd
> > > >>> > > > >>>> AAVMF_CODE.snakeoil.fd
> > > >>> > > > >>>> > > > > > >>> > > AAVMF_VARS.ms.fd
> > > >>> > > > >>>> > > > > > >>> > > AAVMF32_VARS.fd AAVMF_CODE.ms.fd
> > > >>> AAVMF_VARS.fd
> > > >>> > > > >>>> > > > > > >>> > > AAVMF_VARS.snakeoil.fd
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > in my case they have been correctly
> > placed
> > > >>> under
> > > >>> > > > >>>> > > /usr/share/AAVMF
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > I'm not sure that the problem is there.
> > The
> > > >>> error
> > > >>> > > > >>>> message
> > > >>> > > > >>>> > > talks
> > > >>> > > > >>>> > > > > > >>> about the
> > > >>> > > > >>>> > > > > > >>> > > libvirt version that could be wrong.
> > What
> > > >>> about
> > > >>> > > if I
> > > >>> > > > >>>> > > retrocede
> > > >>> > > > >>>> > > > > > >>> libirt 7.0
> > > >>> > > > >>>> > > > > > >>> > > to 6.9 for example. Why 6.9 ?
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > As you can read below,it supports qemu
> > 5.0
> > > >>> and
> > > >>> > > > >>>> newer...
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > v6.9.0 (2020-11-02)
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > -
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > *New features*
> > > >>> > > > >>>> > > > > > >>> > > -
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > nodedev: Add support for channel
> > > >>> subsystem
> > > >>> > > (CSS)
> > > >>> > > > >>>> > > devices on
> > > >>> > > > >>>> > > > > > >>> S390
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > A CSS device is represented as a
> > > >>> parent
> > > >>> > > device
> > > >>> > > > >>>> of a CCW
> > > >>> > > > >>>> > > > > device.
> > > >>> > > > >>>> > > > > > >>> > > This support allows to create
> > vfio-ccw
> > > >>> > > mediated
> > > >>> > > > >>>> devices
> > > >>> > > > >>>> > > > > with
> > > >>> > > > >>>> > > > > > >>> > > virNodeDeviceCreateXML().
> > > >>> > > > >>>> > > > > > >>> > > -
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > qemu: Implement memory failure
> > event
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > New event is implemented that is
> > > >>> emitted
> > > >>> > > > >>>> whenever a
> > > >>> > > > >>>> > > guest
> > > >>> > > > >>>> > > > > > >>> > > encounters a memory failure.
> > > >>> > > > >>>> > > > > > >>> > > -
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > qemu: Implement support for
> > > >>> <transient/>
> > > >>> > > disks
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > VMs based on the QEMU hypervisor
> > now
> > > >>> can use
> > > >>> > > > >>>> > > <transient/>
> > > >>> > > > >>>> > > > > > >>> option
> > > >>> > > > >>>> > > > > > >>> > > for local file-backed disks to
> > > >>> configure a
> > > >>> > > disk
> > > >>> > > > >>>> which
> > > >>> > > > >>>> > > > > discards
> > > >>> > > > >>>> > > > > > >>> changes made
> > > >>> > > > >>>> > > > > > >>> > > to it while the VM was active.
> > > >>> > > > >>>> > > > > > >>> > > -
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > hyperv: implement new APIs
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > The virConnectGetCapabilities(),
> > > >>> > > > >>>> > > virConnectGetMaxVcpus(),
> > > >>> > > > >>>> > > > > > >>> > > virConnectGetVersion(),
> > > >>> > > virDomainGetAutostart(),
> > > >>> > > > >>>> > > > > > >>> > > virDomainSetAutostart(),
> > > >>> > > virNodeGetFreeMemory(),
> > > >>> > > > >>>> > > > > > >>> virDomainReboot(),
> > > >>> > > > >>>> > > > > > >>> > > virDomainReset(),
> > > >>> virDomainShutdown(), and
> > > >>> > > > >>>> > > > > > >>> virDomainShutdownFlags()
> > > >>> > > > >>>> > > > > > >>> > > APIs have been implemented in the
> > > >>> Hyper-V
> > > >>> > > > >>>> driver.
> > > >>> > > > >>>> > > > > > >>> > > -
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > bhyve: implement virtio-9p
> > filesystem
> > > >>> > > support
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > Implement virito-9p shared
> > filesystem
> > > >>> using
> > > >>> > > the
> > > >>> > > > >>>> > > > > <filesystem/>
> > > >>> > > > >>>> > > > > > >>> > > element.
> > > >>> > > > >>>> > > > > > >>> > > -
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > qemu: Add support for vDPA network
> > > >>> devices.
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > VMs using the QEMU hypervisor can
> > now
> > > >>> > > specify
> > > >>> > > > >>>> vDPA
> > > >>> > > > >>>> > > network
> > > >>> > > > >>>> > > > > > >>> devices
> > > >>> > > > >>>> > > > > > >>> > > using <interface type='vdpa'>. The
> > > >>> node
> > > >>> > > device
> > > >>> > > > >>>> APIs
> > > >>> > > > >>>> > > also
> > > >>> > > > >>>> > > > > now
> > > >>> > > > >>>> > > > > > >>> list
> > > >>> > > > >>>> > > > > > >>> > > and provide XML descriptions for
> > vDPA
> > > >>> > > devices.
> > > >>> > > > >>>> > > > > > >>> > > -
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > cpu_map: Add EPYC-Rome CPU model
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > *It's supported in QEMU 5.0.0 and
> > > >>> newer.*
> > > >>> > > > >>>> > > > > > >>> > > -
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > cpu: Add a flag for XML
> > validation in
> > > >>> CPU
> > > >>> > > > >>>> comparison
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > The virConnectCompareCPU and
> > > >>> > > > >>>> > > virConnectCompareHypervisorCPU
> > > >>> > > > >>>> > > > > > >>> API now
> > > >>> > > > >>>> > > > > > >>> > > support the
> > > >>> > > > >>>> VIR_CONNECT_COMPARE_CPU_VALIDATE_XML flag,
> > > >>> > > > >>>> > > > > which
> > > >>> > > > >>>> > > > > > >>> > > enables XML validation. For virsh,
> > > >>> this
> > > >>> > > feature
> > > >>> > > > >>>> is
> > > >>> > > > >>>> > > enabled
> > > >>> > > > >>>> > > > > by
> > > >>> > > > >>>> > > > > > >>> passing the
> > > >>> > > > >>>> > > > > > >>> > > --validate option to the
> > cpu-compare
> > > >>> and
> > > >>> > > > >>>> > > > > hypervisor-cpu-compare
> > > >>> > > > >>>> > > > > > >>> > > subcommands.
> > > >>> > > > >>>> > > > > > >>> > > -
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > qemu: Introduce virtio-balloon
> > free
> > > >>> page
> > > >>> > > > >>>> reporting
> > > >>> > > > >>>> > > feature
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > Introduce the optional attribute
> > > >>> > > > >>>> free-page-reporting
> > > >>> > > > >>>> > > for
> > > >>> > > > >>>> > > > > virtio
> > > >>> > > > >>>> > > > > > >>> > > memballoon device. It
> > > >>> enables/disables the
> > > >>> > > > >>>> ability of
> > > >>> > > > >>>> > > the
> > > >>> > > > >>>> > > > > QEMU
> > > >>> > > > >>>> > > > > > >>> virtio
> > > >>> > > > >>>> > > > > > >>> > > memory balloon to return unused
> > pages
> > > >>> back
> > > >>> > > to
> > > >>> > > > >>>> the
> > > >>> > > > >>>> > > > > hypervisor.
> > > >>> > > > >>>> > > > > > >>> QEMU 5.1 and
> > > >>> > > > >>>> > > > > > >>> > > newer support this feature.
> > > >>> > > > >>>> > > > > > >>> > > -
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > *Improvements*
> > > >>> > > > >>>> > > > > > >>> > > -
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > qemu: Make 'cbitpos' &
> > > >>> 'reducedPhysBits'
> > > >>> > > attrs
> > > >>> > > > >>>> optional
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > Libvirt probes the underlying
> > > >>> platform in
> > > >>> > > order
> > > >>> > > > >>>> to
> > > >>> > > > >>>> > > fill in
> > > >>> > > > >>>> > > > > > >>> these
> > > >>> > > > >>>> > > > > > >>> > > SEV attributes automatically
> > before
> > > >>> > > launching a
> > > >>> > > > >>>> guest.
> > > >>> > > > >>>> > > > > > >>> > > -
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > util: support device stats
> > collection
> > > >>> for
> > > >>> > > > >>>> SR-IOV VF
> > > >>> > > > >>>> > > hostdev
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > For SR-IOV VF hostdevs, libvirt
> > now
> > > >>> supports
> > > >>> > > > >>>> retrieving
> > > >>> > > > >>>> > > > > device
> > > >>> > > > >>>> > > > > > >>> > > traffic stats via the
> > > >>> > > virDomainInterfaceStats
> > > >>> > > > >>>> API and
> > > >>> > > > >>>> > > virsh
> > > >>> > > > >>>> > > > > > >>> > > domifstat.
> > > >>> > > > >>>> > > > > > >>> > > -
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > logging: Allow disabling log
> > rollover
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > Set max_len=0 in virtlogd.conf to
> > > >>> disable
> > > >>> > > log
> > > >>> > > > >>>> rollover.
> > > >>> > > > >>>> > > > > > >>> > > -
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > qemu: Set noqueue qdisc for TAP
> > > >>> devices
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > Set noqueue instead of the former
> > > >>> pfifo_fast
> > > >>> > > > >>>> queue
> > > >>> > > > >>>> > > > > discipline
> > > >>> > > > >>>> > > > > > >>> for
> > > >>> > > > >>>> > > > > > >>> > > TAP devices. It will avoid
> > needless
> > > >>> cost of
> > > >>> > > > >>>> host CPU
> > > >>> > > > >>>> > > cycles
> > > >>> > > > >>>> > > > > > >>> and thus
> > > >>> > > > >>>> > > > > > >>> > > improve performance.
> > > >>> > > > >>>> > > > > > >>> > > -
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > qemu: virtiofs can be used without
> > > >>> NUMA
> > > >>> > > nodes
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > Virtiofs is supported for the VM
> > > >>> without
> > > >>> > > NUMA
> > > >>> > > > >>>> nodes but
> > > >>> > > > >>>> > > > > > >>> configured
> > > >>> > > > >>>> > > > > > >>> > > with shared memory.
> > > >>> > > > >>>> > > > > > >>> > > -
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > *Bug fixes*
> > > >>> > > > >>>> > > > > > >>> > > -
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > hyperv: ensure WQL queries work
> > in all
> > > >>> > > locales
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > Relying on the "Description" field
> > > >>> caused
> > > >>> > > > >>>> queries to
> > > >>> > > > >>>> > > fail
> > > >>> > > > >>>> > > > > on
> > > >>> > > > >>>> > > > > > >>> > > non-"en-US" systems. The queries
> > have
> > > >>> been
> > > >>> > > > >>>> updated to
> > > >>> > > > >>>> > > avoid
> > > >>> > > > >>>> > > > > > >>> using localized
> > > >>> > > > >>>> > > > > > >>> > > strings.
> > > >>> > > > >>>> > > > > > >>> > > -
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > rpc: Fix virt-ssh-helper detection
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > libvirt 6.8.0 failed to correctly
> > > >>> detect the
> > > >>> > > > >>>> > > availability
> > > >>> > > > >>>> > > > > of
> > > >>> > > > >>>> > > > > > >>> the
> > > >>> > > > >>>> > > > > > >>> > > new virt-ssh-helper command on the
> > > >>> remote
> > > >>> > > host,
> > > >>> > > > >>>> and
> > > >>> > > > >>>> > > thus
> > > >>> > > > >>>> > > > > always
> > > >>> > > > >>>> > > > > > >>> > > used the fallback instead; this
> > has
> > > >>> now been
> > > >>> > > > >>>> fixed.
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > What do you think ? Can you share some
> > > >>> > > documentation
> > > >>> > > > >>>> about
> > > >>> > > > >>>> > > how to
> > > >>> > > > >>>> > > > > > >>> > > recompile an older version of libvirt
> > from
> > > >>> source
> > > >>> > > > >>>> code ?
> > > >>> > > > >>>> > > thanks.
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > On Tue, Aug 22, 2023 at 3:35 PM Pavel
> > > >>> Hrdina <
> > > >>> > > > >>>> > > phrdina@xxxxxxxxxx
> > > >>> > > > >>>> > > > > >
> > > >>> > > > >>>> > > > > > >>> wrote:
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > >> On Tue, Aug 22, 2023 at 02:49:05PM
> > +0200,
> > > >>> Mario
> > > >>> > > > >>>> Marietto
> > > >>> > > > >>>> > > wrote:
> > > >>> > > > >>>> > > > > > >>> > >> > Hello to everyone.
> > > >>> > > > >>>> > > > > > >>> > >> >
> > > >>> > > > >>>> > > > > > >>> > >> > I'm trying to use qemu 5.1 with
> > > >>> virt-manager
> > > >>> > > and
> > > >>> > > > >>>> libvirt
> > > >>> > > > >>>> > > on
> > > >>> > > > >>>> > > > > my ARM
> > > >>> > > > >>>> > > > > > >>> > >> > chromebook (armhf 32 bit cpu) running
> > > >>> with
> > > >>> > > Devuan
> > > >>> > > > >>>> 4 as
> > > >>> > > > >>>> > > host
> > > >>> > > > >>>> > > > > o.s.
> > > >>> > > > >>>> > > > > > >>> > >> >
> > > >>> > > > >>>> > > > > > >>> > >> > By default it uses qemu and its
> > > >>> > > > >>>> dependencies,version 5.2.
> > > >>> > > > >>>> > > I
> > > >>> > > > >>>> > > > > > >>> remember
> > > >>> > > > >>>> > > > > > >>> > >> that I
> > > >>> > > > >>>> > > > > > >>> > >> > can't use qemu 5.2,because it doesn't
> > > >>> have any
> > > >>> > > > >>>> support for
> > > >>> > > > >>>> > > > > KVM as
> > > >>> > > > >>>> > > > > > >>> you
> > > >>> > > > >>>> > > > > > >>> > >> can
> > > >>> > > > >>>> > > > > > >>> > >> > read here :
> > > >>> > > > >>>> > > > > > >>> > >> >
> > > >>> > > > >>>> > > > > > >>> > >> >
> > > >>> > > > >>>> > > > > > >>> > >> >
> > > >>> > > > >>>> > > > > > >>>
> > > >>> > > > >>>> > >
> > > >>> > >
> > https://lists.gnu.org/archive/html/qemu-devel/2020-09/msg02074.html
> > > >>> > > > >>>> > > > > > >>> > >> >
> > > >>> > > > >>>> > > > > > >>> > >> >
> > > >>> > > > >>>> > > > > > >>> > >> > For this reason,I've compiled qemu
> > 5.1
> > > >>> from
> > > >>> > > > >>>> source. Below
> > > >>> > > > >>>> > > I
> > > >>> > > > >>>> > > > > shown
> > > >>> > > > >>>> > > > > > >>> how I
> > > >>> > > > >>>> > > > > > >>> > >> > have configured everything such as a
> > > >>> little
> > > >>> > > piece
> > > >>> > > > >>>> of
> > > >>> > > > >>>> > > > > compilation
> > > >>> > > > >>>> > > > > > >>> > >> messages :
> > > >>> > > > >>>> > > > > > >>> > >> >
> > > >>> > > > >>>> > > > > > >>> > >> >
> > > >>> > > > >>>> > > > > > >>> > >> > # apt install libgtk-3-dev
> > libpulse-dev
> > > >>> > > libgbm-dev
> > > >>> > > > >>>> > > > > > >>> libspice-protocol-dev
> > > >>> > > > >>>> > > > > > >>> > >> > libspice-server-dev libusb-1.0-0-dev
> > > >>> > > libepoxy-dev
> > > >>> > > > >>>> > > > > > >>> > >> >
> > > >>> > > > >>>> > > > > > >>> > >> >
> > > >>> > > > >>>> > > > > > >>> > >> > # cp
> > > >>> > > > >>>> /root/Desktop/qemu-v5.1.0/arm-softmmu/qemu-system-arm
> > > >>> > > > >>>> > > > > > >>> /usr/bin
> > > >>> > > > >>>> > > > > > >>> > >> >
> > > >>> > > > >>>> > > > > > >>> > >> > # CFLAGS=-Wno-error ./configure
> > > >>> > > > >>>> > > --target-list=x86_64-softmmu
> > > >>> > > > >>>> > > > > > >>> > >> --enable-opengl
> > > >>> > > > >>>> > > > > > >>> > >> > --enable-gtk --enable-kvm
> > > >>> --enable-guest-agent
> > > >>> > > > >>>> > > --enable-spice
> > > >>> > > > >>>> > > > > > >>> > >> --audio-drv-
> > > >>> > > > >>>> > > > > > >>> > >> > list="oss pa" --enable-libusb
> > > >>> > > > >>>> > > > > > >>> > >> >
> > > >>> > > > >>>> > > > > > >>> > >> >
> > > >>> > > > >>>> > > > > > >>> > >> > A little piece of the log messages
> > that
> > > >>> I've
> > > >>> > > got
> > > >>> > > > >>>> from the
> > > >>> > > > >>>> > > > > > >>> compilation of
> > > >>> > > > >>>> > > > > > >>> > >> > qemu 5.1 :
> > > >>> > > > >>>> > > > > > >>> > >> >
> > > >>> > > > >>>> > > > > > >>> > >> >
> > > >>> > > > >>>> > > > > > >>> > >> >
> > > >>> https://pastebin.ubuntu.com/p/8DYfgPvhXy/
> > > >>> > > > >>>> > > > > > >>> > >> >
> > > >>> > > > >>>> > > > > > >>> > >> >
> > > >>> > > > >>>> > > > > > >>> > >> > These are the resulting versions of
> > my
> > > >>> > > frankenstein
> > > >>> > > > >>>> > > operation
> > > >>> > > > >>>> > > > > :
> > > >>> > > > >>>> > > > > > >>> > >> >
> > > >>> > > > >>>> > > > > > >>> > >> >
> > > >>> > > > >>>> > > > > > >>> > >> > # virsh version
> > > >>> > > > >>>> > > > > > >>> > >> >
> > > >>> > > > >>>> > > > > > >>> > >> > Compiled against library: libvirt
> > 7.0.0
> > > >>> > > > >>>> > > > > > >>> > >> > Using library: libvirt 7.0.0
> > > >>> > > > >>>> > > > > > >>> > >> > Using API: QEMU 7.0.0
> > > >>> > > > >>>> > > > > > >>> > >> > Running hypervisor: QEMU 5.1.0
> > > >>> > > > >>>> > > > > > >>> > >> >
> > > >>> > > > >>>> > > > > > >>> > >> >
> > > >>> > > > >>>> > > > > > >>> > >> > At this point I ran virt-manager. It
> > has
> > > >>> been
> > > >>> > > able
> > > >>> > > > >>>> to
> > > >>> > > > >>>> > > detect
> > > >>> > > > >>>> > > > > > >>> qemu,but I
> > > >>> > > > >>>> > > > > > >>> > >> get
> > > >>> > > > >>>> > > > > > >>> > >> > the following error :
> > > >>> > > > >>>> > > > > > >>> > >> >
> > > >>> > > > >>>> > > > > > >>> > >> >
> > > >>> > > > >>>> > > > > > >>> > >> > Warning : Failed to set up UEFI.
> > > >>> > > > >>>> > > > > > >>> > >> > The Libvirt version does not support
> > > >>> UEFI.
> > > >>> > > > >>>> > > > > > >>> > >> > Install options are limited.
> > > >>> > > > >>>> > > > > > >>> > >> >
> > > >>> > > > >>>> > > > > > >>> > >> > (I have also tried upgrading devuan 4
> > > >>> with
> > > >>> > > devuan
> > > >>> > > > >>>> 5 and
> > > >>> > > > >>>> > > I've
> > > >>> > > > >>>> > > > > got
> > > >>> > > > >>>> > > > > > >>> the
> > > >>> > > > >>>> > > > > > >>> > >> same
> > > >>> > > > >>>> > > > > > >>> > >> > error :
> > > >>> > > > >>>> > > > > > >>> > >>
> > > >>> > > > >>>> > > > > > >>> > >> You most likely need to install
> > > >>> qemu-efi-arm
> > > >>> > > package
> > > >>> > > > >>>> which
> > > >>> > > > >>>> > > > > should
> > > >>> > > > >>>> > > > > > >>> > >> provide 32bit arm firmware files. The
> > > >>> package
> > > >>> > > name
> > > >>> > > > >>>> is a bit
> > > >>> > > > >>>> > > > > > >>> confusing
> > > >>> > > > >>>> > > > > > >>> > >> as it doesn't originate from qemu
> > project,
> > > >>> it is
> > > >>> > > > >>>> from edk2
> > > >>> > > > >>>> > > > > project.
> > > >>> > > > >>>> > > > > > >>> > >>
> > > >>> > > > >>>> > > > > > >>> > >> Without this package libvirt most
> > likely
> > > >>> doesn't
> > > >>> > > > >>>> report any
> > > >>> > > > >>>> > > efi
> > > >>> > > > >>>> > > > > > >>> files
> > > >>> > > > >>>> > > > > > >>> > >> and that's what causes the error you
> > are
> > > >>> hitting.
> > > >>> > > > >>>> > > > > > >>> > >>
> > > >>> > > > >>>> > > > > > >>> > >> Pavel
> > > >>> > > > >>>> > > > > > >>> > >>
> > > >>> > > > >>>> > > > > > >>> > >> >
> > > >>> > > > >>>> > > > > > >>> > >> >
> > > >>> > > > >>>> > > > > > >>> > >> > root@devuan:/usr/bin# virsh version
> > > >>> > > > >>>> > > > > > >>> > >> >
> > > >>> > > > >>>> > > > > > >>> > >> > Compiled against library: libvirt
> > 9.0.0
> > > >>> > > > >>>> > > > > > >>> > >> > Using library: libvirt 9.0.0
> > > >>> > > > >>>> > > > > > >>> > >> > Using API: QEMU 9.0.0
> > > >>> > > > >>>> > > > > > >>> > >> > Running hypervisor: QEMU 5.1.0
> > > >>> > > > >>>> > > > > > >>> > >> >
> > > >>> > > > >>>> > > > > > >>> > >> >
> > > >>> > > > >>>> > > > > > >>> > >> > If I change qemu-system-arm vers. 5.1
> > > >>> with
> > > >>> > > > >>>> qemu-system-arm
> > > >>> > > > >>>> > > > > > >>> 5.2,the error
> > > >>> > > > >>>> > > > > > >>> > >> > disappears. So,it seems that libvirt
> > > >>> does not
> > > >>> > > > >>>> accept
> > > >>> > > > >>>> > > > > > >>> qemu-system-arm
> > > >>> > > > >>>> > > > > > >>> > >> vers.
> > > >>> > > > >>>> > > > > > >>> > >> > 5.1 or maybe any version lower than
> > > >>> 5.2,I don't
> > > >>> > > > >>>> know. But
> > > >>> > > > >>>> > > as
> > > >>> > > > >>>> > > > > I've
> > > >>> > > > >>>> > > > > > >>> said,I
> > > >>> > > > >>>> > > > > > >>> > >> > can't use any version of qemu
> > greater or
> > > >>> equal
> > > >>> > > to
> > > >>> > > > >>>> 5.2 on
> > > >>> > > > >>>> > > my
> > > >>> > > > >>>> > > > > > >>> setup. And I
> > > >>> > > > >>>> > > > > > >>> > >> > want to use virt-manager and libvirt
> > > >>> because I
> > > >>> > > > >>>> find these
> > > >>> > > > >>>> > > > > tools
> > > >>> > > > >>>> > > > > > >>> very
> > > >>> > > > >>>> > > > > > >>> > >> > comfortable instead of using the
> > "raw"
> > > >>> qemu
> > > >>> > > > >>>> parameters. Is
> > > >>> > > > >>>> > > > > there a
> > > >>> > > > >>>> > > > > > >>> > >> > workaround ? Maybe I can recompile
> > > >>> virt-manager
> > > >>> > > > >>>> and / or
> > > >>> > > > >>>> > > > > libvirt
> > > >>> > > > >>>> > > > > > >>> from
> > > >>> > > > >>>> > > > > > >>> > >> the
> > > >>> > > > >>>> > > > > > >>> > >> > source code ? but how ? Do you think
> > > >>> that it
> > > >>> > > could
> > > >>> > > > >>>> work
> > > >>> > > > >>>> > > if I
> > > >>> > > > >>>> > > > > use
> > > >>> > > > >>>> > > > > > >>> > >> something
> > > >>> > > > >>>> > > > > > >>> > >> > like this (if it exists and if it
> > can be
> > > >>> > > reached
> > > >>> > > > >>>> in some
> > > >>> > > > >>>> > > way)
> > > >>> > > > >>>> > > > > :
> > > >>> > > > >>>> > > > > > >>> > >> >
> > > >>> > > > >>>> > > > > > >>> > >> >
> > > >>> > > > >>>> > > > > > >>> > >> > Compiled against library: libvirt
> > 5.0.0
> > > >>> > > > >>>> > > > > > >>> > >> > Using library: libvirt 5.0.0
> > > >>> > > > >>>> > > > > > >>> > >> > Using API: QEMU 5.0.0
> > > >>> > > > >>>> > > > > > >>> > >> > Running hypervisor: QEMU 5.1.0
> > > >>> > > > >>>> > > > > > >>> > >> >
> > > >>> > > > >>>> > > > > > >>> > >> >
> > > >>> > > > >>>> > > > > > >>> > >> > thanks.
> > > >>> > > > >>>> > > > > > >>> > >> >
> > > >>> > > > >>>> > > > > > >>> > >> > --
> > > >>> > > > >>>> > > > > > >>> > >> > Mario.
> > > >>> > > > >>>> > > > > > >>> > >>
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> > > --
> > > >>> > > > >>>> > > > > > >>> > > Mario.
> > > >>> > > > >>>> > > > > > >>> > >
> > > >>> > > > >>>> > > > > > >>> >
> > > >>> > > > >>>> > > > > > >>> >
> > > >>> > > > >>>> > > > > > >>> > --
> > > >>> > > > >>>> > > > > > >>> > Mario.
> > > >>> > > > >>>> > > > > > >>>
> > > >>> > > > >>>> > > > > > >>
> > > >>> > > > >>>> > > > > > >>
> > > >>> > > > >>>> > > > > > >> --
> > > >>> > > > >>>> > > > > > >> Mario.
> > > >>> > > > >>>> > > > > > >>
> > > >>> > > > >>>> > > > > > >
> > > >>> > > > >>>> > > > > > >
> > > >>> > > > >>>> > > > > > > --
> > > >>> > > > >>>> > > > > > > Mario.
> > > >>> > > > >>>> > > > > > >
> > > >>> > > > >>>> > > > > >
> > > >>> > > > >>>> > > > > >
> > > >>> > > > >>>> > > > > > --
> > > >>> > > > >>>> > > > > > Mario.
> > > >>> > > > >>>> > > > >
> > > >>> > > > >>>> > > >
> > > >>> > > > >>>> > > >
> > > >>> > > > >>>> > > > --
> > > >>> > > > >>>> > > > Mario.
> > > >>> > > > >>>> > >
> > > >>> > > > >>>> >
> > > >>> > > > >>>> >
> > > >>> > > > >>>> > --
> > > >>> > > > >>>> > Mario.
> > > >>> > > > >>>>
> > > >>> > > > >>>
> > > >>> > > > >>>
> > > >>> > > > >>> --
> > > >>> > > > >>> Mario.
> > > >>> > > > >>>
> > > >>> > > > >>
> > > >>> > > > >>
> > > >>> > > > >> --
> > > >>> > > > >> Mario.
> > > >>> > > > >>
> > > >>> > > > >
> > > >>> > > > >
> > > >>> > > > > --
> > > >>> > > > > Mario.
> > > >>> > > > >
> > > >>> > > >
> > > >>> > > >
> > > >>> > > > --
> > > >>> > > > Mario.
> > > >>> > >
> > > >>> > >
> > > >>> > >
> > > >>> >
> > > >>> > --
> > > >>> > Mario.
> > > >>>
> > > >>
> > > >>
> > > >> --
> > > >> Mario.
> > > >>
> > > >
> > > >
> > > > --
> > > > Mario.
> > > >
> > >
> > >
> > > --
> > > Mario.
> >
>
>
> --
> Mario.
--Mario.
--Mario.
--Mario.
--
Mario.
