On Sat, 2011-11-19 at 11:17 -0800, Chris Wright wrote: > > Hm, it seems I lied to you about this. The non-coherent mode isn't just > > a historical mistake; it's configurable by the BIOS, and we actually > > encourage people to use the non-coherent mode because it makes the > > hardware page-walk faster — so reduces the latency for IOTLB misses. > > Interesting because for the workloads I've tested it's the exact opposite. > Tested w/ BIOS enabling and disabling coherency, and w/ non-coherent > access and streaming DMA (i.e. bare metal NIC bw testing)...the IOMMU > added smth like 10% when non-coherent vs. coherent. Right. I specifically said "latency for IOTLB misses". That is the main reason we apparently advise BIOS authors to use non-coherent mode. (Of course, the fact that we're letting the BIOS control this at all means that we obviously haven't learned *any* lessons from the last few years of pain with VT-d enabling and BIOS bugs. It should have been under OS control. But that's a separate, stunningly depressing, issue.) Of course, The *overall* performance sucks rocks if we are in non-coherent mode. The bare-metal NIC bandwidth testing is the obvious test case for it, and thanks for providing ballpark numbers for that. They perfectly back up the qualitative remarks I made to Rajesh yesterday on exactly this topic. -- dwmw2
Attachment:
smime.p7s
Description: S/MIME cryptographic signature
