Hi Xiaoyao, On Mon, Jan 15, 2024 at 03:45:58PM +0800, Xiaoyao Li wrote: > Date: Mon, 15 Jan 2024 15:45:58 +0800 > From: Xiaoyao Li <xiaoyao.li@xxxxxxxxx> > Subject: Re: [PATCH v7 10/16] i386/cpu: Introduce cluster-id to X86CPU > > On 1/15/2024 1:59 PM, Zhao Liu wrote: > > (Also cc "machine core" maintainers.) > > > > Hi Xiaoyao, > > > > On Mon, Jan 15, 2024 at 12:18:17PM +0800, Xiaoyao Li wrote: > > > Date: Mon, 15 Jan 2024 12:18:17 +0800 > > > From: Xiaoyao Li <xiaoyao.li@xxxxxxxxx> > > > Subject: Re: [PATCH v7 10/16] i386/cpu: Introduce cluster-id to X86CPU > > > > > > On 1/15/2024 11:27 AM, Zhao Liu wrote: > > > > On Sun, Jan 14, 2024 at 09:49:18PM +0800, Xiaoyao Li wrote: > > > > > Date: Sun, 14 Jan 2024 21:49:18 +0800 > > > > > From: Xiaoyao Li <xiaoyao.li@xxxxxxxxx> > > > > > Subject: Re: [PATCH v7 10/16] i386/cpu: Introduce cluster-id to X86CPU > > > > > > > > > > On 1/8/2024 4:27 PM, Zhao Liu wrote: > > > > > > From: Zhuocheng Ding <zhuocheng.ding@xxxxxxxxx> > > > > > > > > > > > > Introduce cluster-id other than module-id to be consistent with > > > > > > CpuInstanceProperties.cluster-id, and this avoids the confusion > > > > > > of parameter names when hotplugging. > > > > > > > > > > I don't think reusing 'cluster' from arm for x86's 'module' is a good idea. > > > > > It introduces confusion around the code. > > > > > > > > There is a precedent: generic "socket" v.s. i386 "package". > > > > > > It's not the same thing. "socket" vs "package" is just software people and > > > hardware people chose different name. It's just different naming issue. > > > > No, it's a similar issue. Same physical device, different name only. > > > > Furthermore, the topology was introduced for resource layout and silicon > > fabrication, and similar design ideas and fabrication processes are fairly > > consistent across common current arches. Therefore, it is possible to > > abstract similar topological hierarchies for different arches. > > > > > > > > however, here it's reusing name issue while 'cluster' has been defined for > > > x86. It does introduce confusion. > > > > There's nothing fundamentally different between the x86 module and the > > generic cluster, is there? This is the reason that I don't agree with > > introducing "modules" in -smp. > > generic cluster just means the cluster of processors, i.e, a group of > cpus/lps. It is just a middle level between die and core. Not sure if you mean the "cluster" device for TCG GDB? "cluster" device is different with "cluster" option in -smp. When Yanan introduced the "cluster" option in -smp, he mentioned that it is for sharing L2 and L3 tags, which roughly corresponds to our module. > > It can be the module level in intel, or tile level. Further, if per die lp > number increases in the future, there might be more middle levels in intel > between die and core. Then at that time, how to decide what level should > cluster be mapped to? Currently, there're 3 levels defined in SDM which are between die and core: diegrp, tile and module. In our products, L2 is just sharing on the module, so the intel's module and the general cluster are the best match. There are no commercially available machines for the other levels yet, so there's no way to ensure exactly what the future holds, but we should try to avoid fragmentation of the topology hierarchy and try to maintain the uniform and common topology hierarchies for QEMU. Unless a new level for -smp is introduced in the future when an unsolvable problem is raised. > > > > > > > > The direct definition of cluster is the level that is above the "core" > > > > and shares the hardware resources including L2. In this sense, arm's > > > > cluster is the same as x86's module. > > > > > > then, what about intel implements tile level in the future? why ARM's > > > 'cluster' is mapped to 'module', but not 'tile' ? > > > > This depends on the actual need. > > > > Module (for x86) and cluster (in general) are similar, and tile (for x86) > > is used for L3 in practice, so I use module rather than tile to map > > generic cluster. > > > > And, it should be noted that x86 module is mapped to the generic cluster, > > not to ARM's. It's just that currently only ARM is using the clusters > > option in -smp. > > > > I believe QEMU provides the abstract and unified topology hierarchies in > > -smp, not the arch-specific hierarchies. > > > > > > > > reusing 'cluster' for 'module' is just a bad idea. > > > > > > > Though different arches have different naming styles, but QEMU's generic > > > > code still need the uniform topology hierarchy. > > > > > > generic code can provide as many topology levels as it can. each ARCH can > > > choose to use the ones it supports. > > > > > > e.g., > > > > > > in qapi/machine.json, it says, > > > > > > # The ordering from highest/coarsest to lowest/finest is: > > > # @drawers, @books, @sockets, @dies, @clusters, @cores, @threads. > > > > This ordering is well-defined... > > > > > # > > > # Different architectures support different subsets of topology > > > # containers. > > > # > > > # For example, s390x does not have clusters and dies, and the socket > > > # is the parent container of cores. > > > > > > we can update it to > > > > > > # The ordering from highest/coarsest to lowest/finest is: > > > # @drawers, @books, @sockets, @dies, @clusters, @module, @cores, > > > # @threads. > > > > ...but here it's impossible to figure out why cluster is above module, > > and even I can't come up with the difference between cluster and module. > > > > > # > > > # Different architectures support different subsets of topology > > > # containers. > > > # > > > # For example, s390x does not have clusters and dies, and the socket > > > # is the parent container of cores. > > > # > > > # For example, x86 does not have drawers and books, and does not support > > > # cluster. > > > > > > even if cluster of x86 is supported someday in the future, we can remove the > > > ordering requirement from above description. > > > > x86's cluster is above the package. > > > > To reserve this name for x86, we can't have the well-defined topology > > ordering. > > > > But topology ordering is necessary in generic code, and many > > calculations depend on the topology ordering. > > could you point me to the code? Yes, e.g., there're 2 helpers: machine_topo_get_cores_per_socket() and machine_topo_get_threads_per_socket(). > > > > > > > > > > > > > > s390 just added 'drawer' and 'book' in cpu topology[1]. I think we can also > > > > > add a module level for x86 instead of reusing cluster. > > > > > > > > > > (This is also what I want to reply to the cover letter.) > > > > > > > > > > [1] https://lore.kernel.org/qemu-devel/20231016183925.2384704-1-nsg@xxxxxxxxxxxxx/ > > > > > > > > These two new levels have the clear topological hierarchy relationship > > > > and don't duplicate existing ones. > > > > > > > > "book" or "drawer" may correspond to intel's "cluster". > > > > > > > > Maybe, in the future, we could support for arch-specific alias topologies > > > > in -smp. > > > > > > I don't think we need alias, reusing 'cluster' for 'module' doesn't gain any > > > benefit except avoid adding a new field in SMPconfiguration. All the other > > > cluster code is ARM specific and x86 cannot share. > > > > The point is that there is no difference between intel module and general > > cluster...Considering only the naming issue, even AMD has the "complex" to > > correspond to the Intel's "module". > > does complex of AMD really match with intel module? L3 cache is shared in > one complex, while L2 cache is shared in one module for now. If then it could correspond to intel's tile, which is after all a level below die. Thanks, Zhao
