On Sat, Feb 17, 2024 at 04:46:04PM +0000, Michael Kelley wrote: > From: Saurabh Sengar <ssengar@xxxxxxxxxxxxxxxxxxx> Sent: Friday, February 16, 2024 6:10 AM > > To: kys@xxxxxxxxxxxxx; haiyangz@xxxxxxxxxxxxx; wei.liu@xxxxxxxxxx; > > decui@xxxxxxxxxxxxx; linux-hyperv@xxxxxxxxxxxxxxx; linux-kernel@xxxxxxxxxxxxxxx > > Cc: ssengar@xxxxxxxxxxxxx > > Subject: [PATCH] Drivers: hv: Kconfig: select CPUMASK_OFFSTACK for Hyper-V > > > > CPUMASK_OFFSTACK must be set to have NR_CPUS_RANGE_END value greater than > > 512, which eventually allows NR_CPUS > 512. > > > > CPUMASK_OFFSTACK can also be enabled by setting MAXSMP=y, but that will > > set NR_CPUS=8192. This is not accurate for Hyper-V, because maximum number > > of vCPU supported by Hyper-V today is 2048. Thus, enabling MAXSMP increase > > the vmlinux size unnecessary. > > Note that these statements apply only to x86. arm64 doesn't have MAXSMP > or NR_CPUS_RANGE_END. > > > > > This option allows NR_CPUS=2048 which saves around 1MB of vmlinux size > > for Hyper-V. > > > > Signed-off-by: Saurabh Sengar <ssengar@xxxxxxxxxxxxxxxxxxx> > > --- > > drivers/hv/Kconfig | 1 + > > 1 file changed, 1 insertion(+) > > > > diff --git a/drivers/hv/Kconfig b/drivers/hv/Kconfig > > index 0024210..bc3f496 100644 > > --- a/drivers/hv/Kconfig > > +++ b/drivers/hv/Kconfig > > @@ -9,6 +9,7 @@ config HYPERV > > select PARAVIRT > > select X86_HV_CALLBACK_VECTOR if X86 > > select OF_EARLY_FLATTREE if OF > > + select CPUMASK_OFFSTACK > > help > > Select this option to run Linux as a Hyper-V client operating > > system. > > -- > > 1.8.3.1 > > > > I'm not sure that enabling CPUMASK_OFFSTACK for Hyper-V > guests is the right thing to do, as there's additional runtime > cost when CPUMASK_OFFSTACK is enabled. I agree that for > the most general case, you want NR_CPUS to be 2048, which > requires CPUMASK_OFFSTACK. But it would be legitimate to > build a kernel with NR_CPUS set to something like 64 or 256 > for a more limited Hyper-V guest use case, and to not want to > incur the cost of CPUMASK_OFFSTACK. > > You could consider doing something like this: > > select CPUMASK_OFFSTACK if NR_CPUS > 512 Thanks for your review. This was my first thought as well, but for x86, NR_CPUS itself depends on CPUMASK_OFFSTACK and this creates some kind of circular dependency and doesn't work effectively. Here are few key points to note: 1. In ARM64 as well for enabling CPUMASK_OFFSTACK we need to enable DEBUG_PER_CPU_MAPS and that will have additional overhead. This dependency is for all the archs. There was an earlier attempt to decouple it: https://lore.kernel.org/lkml/20220412231508.32629-2-libo.chen@xxxxxxxxxx/ 2. However, for ARM64, NR_CPUS doesn't have dependency on CPUMASK_OFFSTACK. In ARM64 NR_CPUS is quite independent from any policy, we can choose any value for NR_CPUS freely, things are simple. This problem specificaly to be solved for x86. 3. If we have to select more then 512 CPUs on x86, CPUMASK_OFFSTACK needto be enabled, so this additional runtime cost is unavoidable for NR_CPUS > 512. There is no way today to enable CPUMASK_OFFSTACK apart from enabling MAXSMP or DEBUG_PER_CPU_MAPS. Both of these options we don't want to use. I agree that we possibly don't want to enable this option for HyperV VMs where NR_CPUS < 512. I have two thoughts here: 1. Enable it only for VTL platforms, as current requirement for minimal kernel is only for VTL platforms only. 2. Fix this for all of x86. I couldn't find any reson why CPUMASK_OFFSTACK dependency is there on x86 for having more than 512 CPUs. What is special in x86 to have this restriction ? If there is no reason we should relax the restriction of CPUMASK_OFFSTACK for NR_CPUs similar to ARM and other archs. - Saurabh > > But kernel builders always have the option of explicitly > enabling CPUMASK_OFFSTACK. That's what I see in the distro > vendor arm64 images in Azure, since there's currently nothing > that automatically selects CPUMASK_OFFSTACK for arm64. > So I'm wondering if selecting CPUMASK_OFFSTACK under > HYPERV should be added at all. The two aren't really related. > > There are recent LKML threads on enabling CPUMASK_OFFSTACK > for arm64 -- see links below for some useful discussion of the > topic in general. > > Michael > > [1] https://lore.kernel.org/lkml/794a1211-630b-3ee5-55a3-c06f10df1490@xxxxxxxxx/ > [2] https://lore.kernel.org/lkml/7ab6660e-e69f-a64b-0de3-b8dde14f79fa@xxxxxxxxx/ > [3] https://lore.kernel.org/lkml/e0d41efb-a74e-6bb5-f325-63d42358c802@xxxxxxxxxx/
