On Tue, Nov 30, 2021 at 02:32:42PM +0100, Rafael J. Wysocki wrote:
> On Tue, Nov 30, 2021 at 2:22 PM Ricardo Neri
> <ricardo.neri-calderon@xxxxxxxxxxxxxxx> wrote:
> >
> > On Wed, Nov 24, 2021 at 03:48:49PM +0100, Rafael J. Wysocki wrote:
> > > On Sat, Nov 6, 2021 at 2:34 AM Ricardo Neri
> > > <ricardo.neri-calderon@xxxxxxxxxxxxxxx> wrote:
>
> [cut]
>
> > > > +/**
> > > > + * intel_hfi_offline() - Disable HFI on @cpu
> > > > + * @cpu: CPU in which the HFI will be disabled
> > > > + *
> > > > + * Remove @cpu from those covered by its HFI instance.
> > > > + *
> > > > + * On some processors, hardware remembers previous programming settings even
> > > > + * after being reprogrammed. Thus, keep HFI enabled even if all CPUs in the
> > > > + * die/package of @cpu are offline. See note in intel_hfi_online().
> > > > + */
> > > > +void intel_hfi_offline(unsigned int cpu)
> > > > +{
> > > > + struct cpumask *die_cpumask = topology_core_cpumask(cpu);
> > > > + struct hfi_cpu_info *info = &per_cpu(hfi_cpu_info, cpu);
> > > > + struct hfi_instance *hfi_instance;
> > > > +
> > > > + if (!boot_cpu_has(X86_FEATURE_INTEL_HFI))
> > > > + return;
> > > > +
> > > > + hfi_instance = info->hfi_instance;
> > > > + if (!hfi_instance)
> > > > + return;
> > > > +
> > > > + if (!hfi_instance->initialized)
> > > > + return;
> > > > +
> > > > + mutex_lock(&hfi_lock);
> > > > +
> > > > + /*
> > > > + * We were using the core cpumask of @cpu to track CPUs in the same
> > > > + * die/package. Now it is going offline and we need to find another
> > > > + * CPU we can use.
> > > > + */
> > > > + if (die_cpumask == hfi_instance->cpus) {
> > > > + int new_cpu;
> > > > +
> > > > + new_cpu = cpumask_any_but(hfi_instance->cpus, cpu);
> > > > + if (new_cpu >= nr_cpu_ids)
> > > > + /* All other CPUs in the package are offline. */
> > > > + hfi_instance->cpus = NULL;
> > > > + else
> > > > + hfi_instance->cpus = topology_core_cpumask(new_cpu);
> > >
> > > Hmmm. Is topology_core_cpumask() updated when CPUs go offline and online?
> >
> > Yes. A CPU going offline is cleared from its siblings' cpumask [1] and its own [2]
> > in remove_siblinginfo() via cpu_disable_common(). A CPU going online is set
> > in its siblings' cpumask and its own in set_cpu_sibling_map() [3].
>
> OK, so it is necessary to ensure that intel_hfi_offline() will always
> run after remove_siblinginfo() so it sees the updated mask. How do we
> ensure that?
I don't think that is possible. remove_siblinginfo() is called from
CPUHP_TEARDOWN_CPU, which always happens after CPUHP_AP_OFFLINE, if I
understand correctly.
I guess that I will need to use a local cpumask as other drivers do.
Thanks and BR,
Ricardo
