On Wed, Mar 28, 2007 at 08:46:44AM +0800, Zou Nan hai wrote: > On Wed, 2007-03-28 at 03:39, Jack Steiner wrote: > > > This patch adds an optional method for purging the TLB on SN IA64 systems. > > The change should not affect any non-SN system. > > > > Signed-off-by: Jack Steiner <steiner@xxxxxxx> > > > > --- > > > > +void > > +smp_flush_tlb_cpumask (cpumask_t xcpumask) > > +{ > > + unsigned short counts[NR_CPUS]; > > + cpumask_t cpumask = xcpumask; > > + int count, mycpu, cpu, flush_mycpu = 0; > > + > > + preempt_disable(); > > + mycpu = smp_processor_id(); > > + > > + for_each_cpu_mask(cpu, cpumask) { > > + counts[cpu] = per_cpu(local_flush_count, cpu); > > + mb(); > > + if (cpu == mycpu) > > + flush_mycpu = 1; > > + else > > + smp_send_local_flush_tlb(cpu); > > + } > > + > > + if (flush_mycpu) > > + smp_local_flush_tlb(); > > + > > + for_each_cpu_mask(cpu, cpumask) { > > + count = 0; > > + while(counts[cpu] == per_cpu(local_flush_count, cpu)) { > > Due to 64k offset of percpu data, the same percpu variable on different > CPUs are very likely to be on the same cacheline of some levels of > cache. > > So I think the operation on local_flush_count may be very cache > unfriendly... I was concerned about that, too, but testing finally convinced me that it was not an issue. I think the reason is that is takes a few hundred nanoseconds per cpu to send an IPI. So rather than a contended cache line, we have a line that is serially read by multiple cpus. Although contention can occur, typically multiple cpus are not trying to read the same line at the same time. For example (oversimplified), IPI sent to cpu 0 at time 0, to cpu 1 at time ~100, cpu 2 at time ~200, etc. The IPI requires a chipset access that takes order-of-memory-access time. Assume it take N usec for a cpu to recognize the IPI & call the TLB flushing code. Cpu 0 reads local_flush_count at time N, cpu reads local_flush_count at time 100+N, etc. Very little contention, just serial access. -- I tried a second algorithm where the local_flush_count was kept in node-local percpu data. That scheme was significantly slower. Most likely because the cpu that initiates the flush will take N (# of cpus) cache misses to get an initial snapshot of the counts, then another N cache misses to check for completion. This assumes that a cpu doing a flush is not the most-recent cpu to do a flush. I believe this is typical. Keeping the counts in a single array (64cpus/cache line) significantly reduces the number of cache misses. Another disadvantage of keeping counts in per-cpu data is that scanning the counts trashes the TLB for large NR_CPUS. The counts will be located in different 16MB granules. Each reference to cpu's percpu data will require a different TLB entry to map the address used to reference the count. To scan N cpus, there will be ~2*N TLB misses plus at the end of the flush, the contents of the TLB are useless for most kernel or user use. -- I tried a third algorithm where the counts were kept in a single array but each count was cacheline aligned to eliminate any possibility of contention. This was better that the second method that trashed the TLB. 1 TLB entry will cover the entire array. Unfortunately, this algorithm still encurs 2*N cache misses & is slower than the current algorithm. Does this explanation make sense...... If anyone has an alternate algorithm, I be glad to try it. -- jack - To unsubscribe from this list: send the line "unsubscribe linux-ia64" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
![]() |