Re: [PATCH v4 2/5] numa: describe siblings distances within cells

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Fri, 6 Oct 2017 08:49:46 -0600
Jim Fehlig <jfehlig@xxxxxxxx> wrote:

> On 09/08/2017 08:47 AM, Wim Ten Have wrote:
> > From: Wim ten Have <wim.ten.have@xxxxxxxxxx>
> > 
> > Add libvirtd NUMA cell domain administration functionality to
> > describe underlying cell id sibling distances in full fashion
> > when configuring HVM guests.  
> 
> May I suggest wording this paragraph as:
> 
> Add support for describing sibling vCPU distances within a domain's vNUMA cell 
> configuration.

  See below (v5 comment).

> > Schema updates are made to docs/schemas/cputypes.rng enforcing domain
> > administration to follow the syntax below the numa cell id and
> > docs/schemas/basictypes.rng to add "numaDistanceValue".  
> 
> I'm not sure this paragraph is needed in the commit message.
> 
> > A minimum value of 10 representing the LOCAL_DISTANCE as 0-9 are
> > reserved values and can not be used as System Locality Distance Information.
> > A value of 20 represents the default setting of REMOTE_DISTANCE
> > where a maximum value of 255 represents UNREACHABLE.
> > 
> > Effectively any cell sibling can be assigned a distance value where
> > practically 'LOCAL_DISTANCE <= value <= UNREACHABLE'.
> > 
> > [below is an example of a 4 node setup]
> > 
> >    <cpu>
> >      <numa>
> >        <cell id='0' cpus='0' memory='2097152' unit='KiB'>
> >          <distances>
> >            <sibling id='0' value='10'/>
> >            <sibling id='1' value='21'/>
> >            <sibling id='2' value='31'/>
> >            <sibling id='3' value='41'/>
> >          </distances>
> >        </cell>
> >        <cell id='1' cpus='1' memory='2097152' unit='KiB'>
> >          <distances>
> >            <sibling id='0' value='21'/>
> >            <sibling id='1' value='10'/>
> >            <sibling id='2' value='31'/>
> >            <sibling id='3' value='41'/>
> >          </distances>
> >        </cell>
> >        <cell id='2' cpus='2' memory='2097152' unit='KiB'>
> >          <distances>
> >            <sibling id='0' value='31'/>
> >            <sibling id='1' value='21'/>
> >            <sibling id='2' value='10'/>
> >            <sibling id='3' value='21'/>
> >          </distances>
> >        <cell id='3' cpus='3' memory='2097152' unit='KiB'>
> >          <distances>
> >            <sibling id='0' value='41'/>
> >            <sibling id='1' value='31'/>
> >            <sibling id='2' value='21'/>
> >            <sibling id='3' value='10'/>
> >          </distances>
> >        </cell>
> >      </numa>
> >    </cpu>  
> 
> How would this look when having more than one cpu in a cell? I suppose something 
> like
> 
>   <cpu>
>      <numa>
>        <cell id='0' cpus='0-3' memory='2097152' unit='KiB'>
>          <distances>
>            <sibling id='0' value='10'/>
>            <sibling id='1' value='10'/>
>            <sibling id='2' value='10'/>
>            <sibling id='3' value='10'/>
>            <sibling id='4' value='21'/>
>            <sibling id='5' value='21'/>
>            <sibling id='6' value='21'/>
>            <sibling id='7' value='21'/>
>          </distances>
>        </cell>
>        <cell id='1' cpus='4-7' memory='2097152' unit='KiB'>
>          <distances>
>            <sibling id='0' value='21'/>
>            <sibling id='1' value='21'/>
>            <sibling id='2' value='21'/>
>            <sibling id='3' value='21'/>
>            <sibling id='4' value='10'/>
>            <sibling id='5' value='10'/>
>            <sibling id='6' value='10'/>
>            <sibling id='7' value='10'/>
>          </distances>
>       </cell>
>     </numa>
>   </cpu>

  Nope. That machine seems to make a 2 node vNUMA setup. 

  Where;
  * NUMA node(0) defined by <cell id='0'> holds 4 (cores)
    cpus '0-3' with 2GByte of dedicated memory.
  * NUMA node(1) defined by <cell id='1'> holds 4 (cores)
    cpus '4-7' with 2GByte of dedicated memory.

      <cpu>
         <numa>
           <cell id='0' cpus='0-3' memory='2097152' unit='KiB'>
             <distances>
               <sibling id='0' value='10'/>
               <sibling id='1' value='21'/>
             </distances>
           </cell>
           <cell id='1' cpus='4-7' memory='2097152' unit='KiB'>
             <distances>
               <sibling id='0' value='21'/>
               <sibling id='1' value='10'/>
             </distances>
          </cell>
        </numa>
      </cpu>

  Specific configuration would typically report below when examined from
  within the guest domain; (despite ignorance in this example that it
  _DOES_ concern a single socket 8 cpu machine).

      [root@f25 ~]# lscpu
      Architecture:          x86_64
      CPU op-mode(s):        32-bit, 64-bit
      Byte Order:            Little Endian
      CPU(s):                8
      On-line CPU(s) list:   0-7
      Thread(s) per core:    1
      Core(s) per socket:    8
      Socket(s):             1
  *>  NUMA node(s):          2
      Vendor ID:             AuthenticAMD
      CPU family:            21
      Model:                 2
      Model name:            AMD FX-8320E Eight-Core Processor
      Stepping:              0
      CPU MHz:               3210.862
      BogoMIPS:              6421.83
      Virtualization:        AMD-V
      Hypervisor vendor:     Xen
      Virtualization type:   full
      L1d cache:             16K
      L1i cache:             64K
      L2 cache:              2048K
      L3 cache:              8192K
  *>  NUMA node0 CPU(s):     0-3
  *>  NUMA node1 CPU(s):     4-7
      Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm rep_good nopl cpuid extd_apicid pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 x2apic popcnt aes xsave avx f16c hypervisor lahf_lm svm cr8_legacy abm sse4a misalignsse 3dnowprefetch ibs xop lwp fma4 tbm vmmcall bmi1 arat npt lbrv nrip_save tsc_scale vmcb_clean decodeassists pausefilter

      [root@f25 ~]# numactl -H
      available: 2 nodes (0-1)
      node 0 cpus: 0 1 2 3
      node 0 size: 1990 MB
      node 0 free: 1786 MB
      node 1 cpus: 4 5 6 7
      node 1 size: 1950 MB
      node 1 free: 1820 MB
      node distances:
      node   0   1 
        0:  10  21 
        1:  21  10 

> In the V3 thread you mentioned "And to reduce even more we could also
> remove LOCAL_DISTANCES as they make a constant factor where; (cell_id == 
> sibling_id)". In the above example cell_id 1 == sibling_id 1, but it is not 
> LOCAL_DISTANCE.
> 
> > Whenever a sibling id the cell LOCAL_DISTANCE does apply and for any
> > sibling id not being covered a default of REMOTE_DISTANCE is used
> > for internal computations.  
> 
> I'm having a hard time understanding this sentence...

  Me.2

> I didn't look closely at the patch since I'd like to understand how multi-cpu 
> cells are handled before doing so.

  Let me prepare v5.  I found a silly error in code being fixed and
  given above commented confusion like to take a better approach under
  the commit messages and witin the cover letter.

Regards,
- Wim.

--
libvir-list mailing list
libvir-list@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/libvir-list



[Index of Archives]     [Virt Tools]     [Libvirt Users]     [Lib OS Info]     [Fedora Users]     [Fedora Desktop]     [Fedora SELinux]     [Big List of Linux Books]     [Yosemite News]     [KDE Users]     [Fedora Tools]
  Powered by Linux