On Tue, 26 Nov 2024 16:18:26 -0600 Mitchell Augustin <mitchell.augustin@xxxxxxxxxxxxx> wrote: > > The BAR space is walked, faulted, and mapped. I'm sure you're at > > least experiencing scaling issues of that with 128GB BARs. > > The part that is strange to me is that I don't see the initialization > slowdown at all when the GPUs are hotplugged after boot completes. > Isn't what you describe here also happening during the hotplugging > process, or is it different in some way? The only thing that comes to mind would be if you're using a vIOMMU and it's configured in non-passthrough mode so the BARs aren't added to the DMA address space after the vIOMMU is enabled during boot. But your virt-install command doesn't show a vIOMMU configuration for the VM. If the slowness is confined to the guest kernel boot, can you share the log of that boot with timestamps? Thanks, Alex
