On 02/27/2013 12:10 PM, Linus Torvalds wrote:
Ugh. That really is rather random. "short" and fserver seems to improve a lot (including the "new" version), the others look like they are either unchanged or huge regressions. Is there any way to get profiles for the improved versions vs the regressed ones? It might well be that we have two different classes of spinlocks. Maybe we could make the back-off version be *explicit* (ie not part of the normal "spin_lock()", but you'd use a special "spin_lock_backoff()" function for it) because it works well for some cases but not for others?
If we have two classes of spinlocks, I suspect we would be better off making those high-demand spinlocks MCS or LCH locks, which have the property that having N+1 CPUs contend on the lock will never result in slower aggregate throughput than having N CPUs contend. Both the current spinlock code and the spinlock code with backoff has the annoying property that adding more waiters to the lock can slow down aggregate throughput. The backoff code seems to push that point a little further out. It appears that that may result in "lesser bottlenecks" disappearing, which in turn triggers amplification of the worst bottlenecks.
Hmm? At the very least, it would give us an idea of *which* spinlock it is that causes the most pain. I think your earlier indications was that it's the mutex->wait_lock or something?
I can certainly take profiles of various workloads, but there is absolutely no guarantee that I will see the same bottlenecks that eg. the people at HP have seen. The largest test system I currently have access to has 40 cores, vs. the 80 cores in the (much more interesting) HP results I pasted. Would you also be interested in performance numbers (and profiles) of a kernel that has bottleneck spinlocks replaced with MCS locks? That could make for an interesting comparison... -- All rights reversed -- To unsubscribe from this list: send the line "unsubscribe linux-tip-commits" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html