On Wed, Jan 30, 2013 at 4:12 PM, Doug Goldstein <cardoe@xxxxxxxxxx> wrote: > On Wed, Jan 30, 2013 at 12:53 PM, Scott Sullivan > <ssullivan@xxxxxxxxxxxxx> wrote: >> On 01/29/2013 02:55 PM, Doug Goldstein wrote: >>> >>> On Tue, Jan 29, 2013 at 9:52 AM, Scott Sullivan<ssullivan@xxxxxxxxxxxxx> >>> wrote: >>> >>>> >>>> I have a machine with four AMD Opteron(TM) 6272 processors: >>>> >>>> >>>> http://www.amd.com/us/products/server/processors/6000-series-platform/6200/Pages/6200-series-processors.aspx#5 >>>> >>>> I use 'virsh nodeinfo' to determine the amount of physical CPUs, and CPU >>>> cores. What values should I be looking at in the output of nodeinfo to >>>> determine this? With 1.X libvirts the output values have changed from >>>> 0.9.4: >>>> >>>> 0.9.4: >>>> [root@host ~]# virsh nodeinfo >>>> CPU model: x86_64 >>>> CPU(s): 64 >>>> CPU frequency: 2099 MHz >>>> CPU socket(s): 4 >>>> Core(s) per socket: 8 >>>> Thread(s) per core: 2 >>>> NUMA cell(s): 1 >>>> Memory size: 24713980 kB >>>> [root@host28 ~]# >>>> >>>> 1.0.1: >>>> [root@host ~]# virsh nodeinfo >>>> CPU model: x86_64 >>>> CPU(s): 64 >>>> CPU frequency: 2099 MHz >>>> CPU socket(s): 1 >>>> Core(s) per socket: 8 >>>> Thread(s) per core: 2 >>>> NUMA cell(s): 4 >>>> Memory size: 24713980 KiB >>>> [root@host28 ~]# >>>> >>>> As you can see, the CPU socket and NUMA cells values have changed in >>>> these >>>> two versions, on the same machine. To get the physical CPU count, is it >>>> safe >>>> to assume this? >>>> >>>> Numa cell(s) X CPU Socket(s) == physical CPUs present >>>> >>>> However, I am not positive what formula I should use to determine the CPU >>>> core count. On this processor, it has 16 cores. Would that mean I can do >>>> this? >>>> >>>> Core(s) per socket X Thread(s) per core: == CPU cores >>>> >>>> Thanks in advance for clarifying. >>>> >>>> -- >>>> libvir-list mailing list >>>> libvir-list@xxxxxxxxxx >>>> https://www.redhat.com/mailman/listinfo/libvir-list >>>> >>> >>> Scott, >>> >>> I've got the exact same CPUs (Dell PowerEdge R815). You'll find the >>> NUMA info is detected node info is detected incorrectly. Oddly we have >>> different data detected. What kernel are you running? >>> >>> with libvirt 1.0.1 >>> root@host ~ # virsh nodeinfo >>> CPU model: x86_64 >>> CPU(s): 64 >>> CPU frequency: 2100 MHz >>> CPU socket(s): 1 >>> Core(s) per socket: 64 >>> Thread(s) per core: 1 >>> NUMA cell(s): 1 >>> Memory size: 132013200 KiB >>> >>> virsh capabilities is a bit "smarter" but still not 100%. It sees 8 >>> NUMA nodes and puts the CPUs in the correct nodes but still detects >>> the CPU incorrectly. >>> >>> >> >> Doug, >> >> I'm running a RedHat kernel; 2.6.32-220.2.1.el6.x86_64 specifically. >> >> Thanks for the insight of your experiences, very interesting. I find the >> same that the NUMA info appears detected incorrectly for this processor. > > Scott, > > It appears that the CPU info isn't detected correctly because it can't > be detected correctly or really expressed in the values we know/have. > For some info you can see: > https://bugzilla.redhat.com/show_bug.cgi?id=874050 > > The issue is that libvirt and to extent the Linux kernel expect 1 > physical socket to equate to 1 NUMA node. In the case of these > processors there are really 2 individual "packages" in 1 physical > socket. Each package has its own memory controller and that is tied to > 8 cores. Hence how you get 8 NUMA nodes with 8 processors each. > However to add to the confusion AMD has implemented their own type of > hyperthreading in that they have two separate cores but 2 cores share > 1 FPU. So in reality each package has 8 cores but if you're doing FPU > work, you only have 4 per package. > > The previous decision was that 4 sockets, 8 cores per socket and 2 > thread per core was not a correct way to express this CPU since it is > true that each core separate. But the current solution is IMHO worse > by saying 1 socket, 64 cores, 1 thread per core. A better result would > be to say 8 sockets, 8 cores per socket, and 1 thread per core. But > that would require us to break the idea that a socket is a physical > socket and align a socket with a "package". > > Anyone else thoughts? Would this be acceptable? Right now the output > is broken and if you put any NUMA values into the XML or use numad the > domain will not start and will give a cryptic error message. Which is > a lot worst than reporting 4 sockets, 8 cores, 1 thread or 8 sockets, > 8 cores and 1 thread IMHO. > > -- > Doug Goldstein Its worth noting that even with the new capabilities data in 1.0.2, this is still behaving incorrectly when you attempt to start the VM and have ANY numa configuration in the XML for the domain. kvm_init_vcpu failed: Cannot allocate memory -- Doug Goldstein -- libvir-list mailing list libvir-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/libvir-list