On Fri, 2009-02-20 at 02:56 -0800, Ingo Molnar wrote:
> * Suresh Siddha <suresh.b.siddha@xxxxxxxxx> wrote:
>
> > On Thu, 2009-02-19 at 04:20 -0800, Ingo Molnar wrote:
> > > Could you please refresh this patch to latest tip:master? The
> > > APIC drivers moved to arch/x86/kernel/apic/.
> >
> > Appended the refreshed patch. Thanks.
>
> thanks. Two details i noticed:
>
> Firstly:
>
> > +++ b/arch/x86/kernel/apic/x2apic_cluster.c
> > +++ b/arch/x86/kernel/apic/x2apic_phys.c
>
> how about x2apic_uv.c? It uses uv_write_global_mmr64() in its
> IPI sending method, which uses:
>
> static inline void uv_write_global_mmr64(int pnode, unsigned long offset,
> unsigned long val)
> {
> *uv_global_mmr64_address(pnode, offset) = val;
> }
>
> which uses ->mmr_base, which is mapped via:
>
> init_extra_mapping_uc(UV_LOCAL_MMR_BASE, UV_LOCAL_MMR_SIZE);
>
> so it should be fine because uncached - but at minimum we should
> put a comment into x2apic_uv.c that the generic IPI code relies
> on the lowlevel code serializing - i.e. relies on the UC PAT
> attribute.
Oops. I forgot to mention that and copy Jack to confirm our
understanding. I can send a followup patch adding the comments for UV.
Jack, are you ok?
>
> Secondly, you added smp_mb(), which will translate to an MFENCE.
>
> But in theory it should be enough to have a wmb() here. [Note,
> not an smp_wmb() that i suggested before.] That will translate
> to an SFENCE - which will serialize writes but still allows
> reads/prefetches to pass.
>
> So the question is, is an SFENCE there enough to serialize the
> WRMSR with previous memory-writes? It's not specified in the
> x2apic docs as far as i could see.
No. sfence is not enough (wrmsr to x2apic regs was still passing ahead).
We have done a small experiment to demonstrate the issue and adding
mfence fixes the issue but not sfence. We need a serializing instruction
or mfence. I will try to get the SDM updated.
thanks,
suresh
--
To unsubscribe from this list: send the line "unsubscribe linux-arch" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
