Re: NUMA node - Memory Only

[Jin Huang <andy.jinhuang@xxxxxxxxx>]

Hi, Michal

Thank you for your help.

After compiled and installed libvirt-6.7.0, when I try virsh define xmlfile, it says "error: invalid argument: could not find capabilities for arch=x86_64 domaintype=kvm".

The way I compiled and installed libvirt-6.7.0 is according to https://libvirt.org/compiling.html

My system is Ubuntu20, originally I used apt to install the libvirt, it works well, just not support HMAT.

Do you know how to fix my current issue?

Thank You

Best

Jin Huang

On Fri, Aug 5, 2022 at 5:34 AM Michal Prívozník <mprivozn@xxxxxxxxxx> wrote:

On 8/5/22 11:25, Jin Huang wrote:
> Hi, libvirt-users
> How could I set up a memory-only and no-cpu NUMA node for the qemu-VM
> with the XML file?
>
> Seems each NUMA cell has to be bundled with some specific cpuids.
> If I write the element like this, it is wrong.
>
> <cpu mode='host-passthrough'>
>   <numa>
>     <cell id='0' cpus='0-3' memory='16' unit='GiB'/>
>     <cell id='1' cpus='*null*' memory='16' unit='GiB'/>
>   </numa>
> </cpu>
> Also if I ignore the cpus item, it is not acceptable for the virsh command.
>
> The reason I want the memory-only node is that I want to set up multiple
> memory tiers for the VM, just
> like https://stevescargall.com/2022/06/10/using-linux-kernel-memory-tiering/
> <https://stevescargall.com/2022/06/10/using-linux-kernel-memory-tiering/>
>

What you want to configure is so called HMAT (Heterogeneous Memory
Attribute Table). It can be configured by leaving out the @cpus
attribute completely. For instance like this:

<cpu mode='host-model' check='partial'>
    <numa>
      <cell id='0' cpus='0-5' memory='512000' unit='KiB' discard='yes'>
        <distances>
          <sibling id='0' value='10'/>
          <sibling id='1' value='21'/>
        </distances>
        <cache level='3' associativity='direct' policy='writeback'>
          <size value='10' unit='KiB'/>
          <line value='8' unit='B'/>
        </cache>
        <cache level='1' associativity='direct' policy='writeback'>
          <size value='8' unit='KiB'/>
          <line value='5' unit='B'/>
        </cache>
      </cell>
      <cell id='1' memory='512000' unit='KiB'>
        <distances>
          <sibling id='0' value='21'/>
          <sibling id='1' value='10'/>
        </distances>
      </cell>
      <interconnects>
        <latency initiator='0' target='0' cache='1' type='access'
value='5'/>
        <bandwidth initiator='0' target='0' type='access' value='204800'
unit='KiB'/>
      </interconnects>
    </numa>
  </cpu>


This was introduced in libvirt-6.6.0 so you may need to upgrade.

Michal