just learned something new: cacheline ping pong. http://everything2.com/index.pl?node_id=1382347 http://lwn.net/Articles/176024/ and the details is perhaps best described here: http://www.patentstorm.us/patents/5713004/claims.html what happened is the write-back/through cacheline control is imposed on the L1/L2 cache. but the memory barrier stuff u mentioned (which I just learned today....please correct me) is at the memory bus level, to prevent contention among CPUs. The facts are from Intel Optimization manual (chap2), but the link below (page4) has some info too (plus the ping-pong effect): http://morecores.cn/publication/pdf/Computing%20PI%20to%20Understand%20Key%20Issues%20in%20Multi.pdf Have fun!! On Sun, Sep 27, 2009 at 2:05 PM, Peter Teoh <htmldeveloper@xxxxxxxxx> wrote: > On Sun, Sep 27, 2009 at 1:48 AM, Mulyadi Santosa > <mulyadi.santosa@xxxxxxxxx> wrote: >> Hi.... >> >> On 9/26/09, Peter Teoh <htmldeveloper@xxxxxxxxx> wrote: >>> sorry for being a busy body....:-)....but hope i can contribute my 2cts... >>> >>> On Fri, Sep 25, 2009 at 12:54 PM, Mulyadi Santosa >>> <mulyadi.santosa@xxxxxxxxx> wrote: >>>> On 9/25/09, Jason Nymble <jason.nymble@xxxxxxxxx> wrote: >>>>> For posterity, I've finally solved this issue. It ended up having >>>>> nothing to do with the interrupts/tasklets themselves. The driver uses >>>>> ioremap() to get hold of some reserved memory, and it seems from about >>>>> 2.6.25 onwards or so this defaults to ioremap_nocache(), so our driver >>>>> was doing memory operations in the tasklet on uncacheable pages. >>>>> Calling ioremap_cache() explicitly in the driver solved the issue >>>>> (nice when you can fix a perf regression of 50-100x with a single line >>>>> fix!). Oprofile was of tremendous help in solving this issue. >>>> >>>> Hi Jason >>>> >>>> Nice to hear you finally fix your problem. BTW, may I ask a bit about >>>> ioremap_cache? By using this function, it is like saying "map and >>>> cache these pages so next time we access them, we don't need to fetch >>>> them from device memory again"? CMIIW >>> >>> not really about reading.....but writing. and it is not memory >>> caching, but hardware caching (sounds like the same???? :-)) >>> Read the source: >>> >>> 279 void __iomem *ioremap_cache(resource_size_t phys_addr, >>> unsigned long size) >>> 280 { >>> 281 return __ioremap_caller(phys_addr, size, _PAGE_CACHE_WB, >>> 282 __builtin_return_address(0)); >>> 283 } >> >> Ehem...I see.... however, by doing write back and hold it in page >> cache, it also brings advantage to read operation in near future, >> right? Assume we do write and few nanoseconds later we want to read >> it. Instead of reading through slow outer I/O bus, we read it directly > > sort of....instead of reading from RAM....it is read from L2 or L1 > cache. The speed is much faster. > >> from RAM...and it's faster. >> >>> it is set with WRITEBACK attribute, and write-back is a hardware >>> caching feature (search write-back here): >>> >>> http://en.wikipedia.org/wiki/Cache >>> http://en.wikipedia.org/wiki/CPU_cache >>> >>> write-through is synchronous (meaning the CPU will not execute the >>> next instruction, until the data on the memory bus has updated the >>> device in a hardware handshake protocol). but write-back is >>> asynchronous, for performance reason, it try to accumulate changes >>> before batching them in a burst of write-flush. but the delay is >>> usually quite small. >> >> Hm, does it do LOCK on all CPU? i.e something like doing memory >> barriers on all CPU ? >> > > seemed like not so. If u see the figure in the Intel manual, L2 > cache is shared by multicore, but L1 is per-core. And if multiple > core is writing to the L2 cache at the same time, the contents may > overwrite each other (for WB mode). Anyway, the following is > reproduced verbatim: > > page 2-20 of Intel Optimization Reference Manual: > > When an instruction writes data to a memory location that has WB > memory type, the > processor first ensures that the line is in Exclusive or Modified > state in its own DCU. > The processor looks for the cache line in the following locations, in > the specified > order: > 1. DCU of initiating core > 2. DCU of the other core and L2 cache > 3. System memory > The cache line is taken from the DCU of the other core only if it is > modified, ignoring > the cache line availability or state in the L2 cache. After reading > for ownership is > completed, the data is written to the first-level data cache and the > line is marked as > modified. > > another source of info is System Programming manual Vol 3A, chapter 11 > - the whole chapter is on caching. > > >> -- >> regards, >> >> Mulyadi Santosa >> Freelance Linux trainer >> blog: the-hydra.blogspot.com >> > > > > -- > Regards, > Peter Teoh > -- Regards, Peter Teoh -- To unsubscribe from this list: send an email with "unsubscribe kernelnewbies" to ecartis@xxxxxxxxxxxx Please read the FAQ at http://kernelnewbies.org/FAQ
