On Sun, 16 Oct 2005, Aritz Bastida wrote:
* I asked what would happen if I am resetting a counter (actually the
whole struct with memset) in one CPU while the other one is updating
it. What would be the worst thing it could happen?
That some counters are not reset properly.
For example, if the counter is of type long (64 bytes), could it
happen that one CPU updates upper 32 bytes while the other CPU clears
lower 32 bytes, thus corrupting the counter's value?
(I assume you meant bits not bytes above..)
Yes, that is a possible effect on 32-bit systems. Which part gets
corrupted is dependent on endianness.
Any datatype larger than the basic word size of the architecture needs
special care. You can only do completely lockless operations on counters
up to the word size.
No, without locking or serialization by doing the reset from the
per-cpu thread the reset can be lost.
Why would be lost? In any case, the result would be strange, no?
Example:
CPU0 is incrementing the counter
CPU1 is resetting the counter
reg = some cpu register
counter = the counter memory location
CPU0 CPU1
reg = counter
reg = reg + 1 counter = 0
counter = reg
In the above the reset is lost as it is overwritten by the store of the
incremented counter.
If the counter is of a larger type where the hardware does not do atomic
store/write operations then very strange effects can be seen, such as the
half cleared counter you mentioned above.
As we both mentioned to avoid this you do not clear the per-cpu counters
directly. Instead you just set a flag in the per-cpu data indicating that
the counters should be cleared and let the per-cpu thread clear the actual
counters and reset the flag the next time it is run. If you need to read
the per-cpu counters before them, read them as zero if the flag is still
set.
This all mentioned, there is already a percpu_counter primitive in the
Linux kernel (2.6) which tries to make a reasonable tradeoff for common
use. Today that function is pretty much optimized for read-mostly counters
on either UP-systems (then reduces to a plain counter) or large scale SMP
systems with very many CPUs. This primitive gives you a jitter
exponential to the number of CPUs you have.
The Linux kernel per_cpu_counter implementation is perhaps slightly more
complex than it needs to be for the more common small to medium sized SMP
systems, and also not optimal for write mostly counters. But quite good
balanced for general purpose.
The Linux kernel percpu_counter is defined in
include/linux/percpu_counter.h
Regards
Henrik
-
: send the line "unsubscribe linux-net" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
