Hello everyone!
I am writing a "network analyser" system in kernel space, which is
designed to work in SMP platforms. The main idea is to capture and
analyse the packets in kernel space without the need to context-switch
to user space. I am using Linux 2.6.13 and network drivers with NAPI
enabled. What I do is, basically, capture the packets (skb) in
netif_receive_skb() and in this function insert the packet in a queue.
In parallel, there are as many kernel threads as CPUs (bound to their
respective CPU). They extract the packets from the queue and process
them in parallel.
As all this is done in parallel, with a kernel thread for each online
cpu, I thought that statistics should be per-cpu. Something like:
struct pcap_stats {
unsigned int captured;
unsigned int processed;
unsigned int dropped;
...
};
DEFINE_PER_CPU(struct pcap_stats, pcap_stats);
As I have read in Linux Device Drivers 3rd, this is what it's done in
the networking code of Linux, to avoid locking each time those
variables are modified, and to improve cache performance. LDD3 says:
"Rather than deal with the caching and locking issues, the networking
developers put the statiscts counters into per-CPU variables. Updates
are now lockless and fast. On the rare occasion that user space
requests to see the values of the counters, it is a simple matter to
add up each processor's version and return the total."
Ok, I've done that. But now I have some questions about this:
1.- Suppose that I want to read the statistics (per-cpu and the total
numbers) via proc files. The proc read function would have to access
other cpu's variables, which in turn, could be updated by its cpu at
the same time. So i guess that, even if i'm just going to read the
counters (so there is no corruption of data), some kind of locking
would be required. But if I do that, then why did I need per-cpu
variables? Weren't they for avoiding locks?
2.- Suppose that I also want to be able to reset the counters from
user space (via proc files too). Now I would be changing the counters
in the proc write function. This time the data could get corrupted
because another cpu could be updating its data at the same time. So
again locking would be required. Note that I just want to reset the
counters, not anything strange or difficult (I guess that even if some
other cpu would be incrementing one counter, instead of resetting to
cero, its value would be one, or something similar).
Thank you for your time.
Bye!
Aritz Bastida
-
: send the line "unsubscribe linux-net" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
