Hi David,
On Mon, Jul 14, 2014 at 6:19 PM, David Rientjes <rientjes@xxxxxxxxxx> wrote:
>
> On Sat, 12 Jul 2014, Jiri Kosina wrote:
>
> > I am pretty sure I've seen ppc64 machine with memoryless NUMA node.
> >
>
> Yes, Nishanth Aravamudan (now cc'd) has been working diligently on the
> problems that have been encountered, including problems in generic kernel
> code, on powerpc with memoryless nodes.
Thanks for Cc'ing me on this discussion. I'm going to review Jiang's patchset now, as best I can, but yes I can confirm we see memoryless nodes somewhat frequently on powerpc under PowerVM, due to presumably hypervisor fragmentation (the reason isn't clear to an LPAR, as it's just given a topology).
I agree with Dave Hansen that this seems like a "good thing" to try and figure out, unless KVM decides it's going to hide the underlying topology of a guest's memory from the guest -- which I think could lead (eventually) to confusing performance results.
I believe I have also seen them in hardware on ia64 (cpu-only and memory-only drawers), but not sure if those specific models are in production still.
Finally, I will say that in working on supporting memoryless nodes, I've come across what look like bugs in the NUMA code. Or more accurately, assumptions which aren't always true. So it's a useful exercise for that reason to.
Thanks,
Nish
On Mon, Jul 14, 2014 at 6:19 PM, David Rientjes <rientjes@xxxxxxxxxx> wrote:
>
> On Sat, 12 Jul 2014, Jiri Kosina wrote:
>
> > I am pretty sure I've seen ppc64 machine with memoryless NUMA node.
> >
>
> Yes, Nishanth Aravamudan (now cc'd) has been working diligently on the
> problems that have been encountered, including problems in generic kernel
> code, on powerpc with memoryless nodes.
Thanks for Cc'ing me on this discussion. I'm going to review Jiang's patchset now, as best I can, but yes I can confirm we see memoryless nodes somewhat frequently on powerpc under PowerVM, due to presumably hypervisor fragmentation (the reason isn't clear to an LPAR, as it's just given a topology).
I agree with Dave Hansen that this seems like a "good thing" to try and figure out, unless KVM decides it's going to hide the underlying topology of a guest's memory from the guest -- which I think could lead (eventually) to confusing performance results.
I believe I have also seen them in hardware on ia64 (cpu-only and memory-only drawers), but not sure if those specific models are in production still.
Finally, I will say that in working on supporting memoryless nodes, I've come across what look like bugs in the NUMA code. Or more accurately, assumptions which aren't always true. So it's a useful exercise for that reason to.
Thanks,
Nish