On Tue, Jan 16, 2024 1:08:20PM +0000, Mark Rutland wrote: > On Mon, Jan 15, 2024 at 03:39:00PM +0000, Russell King (Oracle) wrote: > > On Thu, Dec 14, 2023 at 04:05:56PM -0800, Christoph Lameter (Ampere) wrote: > > > +# Determines the placement of cpumasks. > > > +# > > > +# With CPUMASK_OFFSTACK the cpumasks are dynamically allocated. > > > +# Useful for machines with lots of core because it avoids increasing > > > +# the size of many of the data structures in the kernel. > > > +# > > > +# If this is off then the cpumasks have a static sizes and are > > > +# embedded within data structures. > > > +# > > > +config CPUMASK_OFFSTACK > > > + def_bool y > > > + depends on NR_CPUS > 256 > > > > Should that be ">= 256" ? > > I don't think that ">= 256" makes sense. Note that since the cpumasks are > arrays of unsigned long, they're chunked into groups of 64 bits: > > 2 to 64 cpus: 1 x unsigned long => 8 bytes > 65 to 128 cpus: 2 x unsigned long => 16 bytes > 129 to 192 cpus: 3 x unsigned long => 24 bytes > 193 to 256 cpus: 4 x unsigned long => 32 bytes > 257 to 320 cpus: 5 x unsigned long => 40 bytes > > ... and so if a mask for 256 CPUs is too big to go in the stack, so is any mask > for 193+ CPUs, and so ">= 256" should be clamped down to ">= 193" or "> 192". > The boundary should be just after a multiple of 64. > > How did we choose 256 specifically? I note that x86-64 allows 512 CPUs before > requiring CPUMASK_OFFSTACK, and I see that powerpc selects CPUMASK_OFFSTACK > when NR_CPUS >= 8192. > > Mark. The suggestion for >= 256 may have been a zero-index/one-index mixup. It seems > 256 was chosen as the cutoff simply because it preserves existing behavior. The patch description seems to imply there was pushback from distro maintainers on just increasing the default NR_CPUS. The existing default value of 256 is probably already a strain for smaller ARM systems, to which x86-64 isn't a reasonable comparison. I'm not sure what the reaction to increasing from 64 to 256 in 2019 was like, but picking a pivot point for CPUMASK_OFFSTACK beyond 256 may skew the balance even less in favor of smaller ARM systems.
