I've been working on a problem that appears to be due to the way the
socket buffer is managed. I'm using 2.4.20 with SMP on IA-64, but the
code is still the same in 2.6. Here's the details:
When using AF_UNIX/SOCK_STREAMS, and very small messages (20 bytes), the
buffer fills up after about ~180 messages (or ~3600 bytes), even though
the buffer is 64K. I've found that IA-64 SMP will align to a 128 byte
boundary (the size of L1 cache). Non-SMP will align to 8 bytes. This
accounts for part of the problem, since the minimum size is actually 128
bytes, making 180 message actually use 23040 bytes. After reading over
the code, I discover that not only is the data in the buffer accounted
for, but also the overhead. This could make sense, especially in small
memory systems where you would need to keep track of every byte. What's
really odd is what the code actually does. Here's a snippet from
alloc_skb() in skbuff.c:
/* Get the DATA. Size must match skb_add_mtu(). */
size = SKB_DATA_ALIGN(size);
data = kmalloc(size + sizeof(struct skb_shared_info), gfp_mask);
if (data == NULL)
goto nodata;
/* XXX: does not include slab overhead */
skb->truesize = size + sizeof(struct sk_buff);
Note: SKB_DATA_ALIGN is where the 128 byte round up occurs
If you notice, the kmalloc grabs enough space for the data (size) and
the shared info structure, but the truesize field is set to the data
size and the size of the sk_buff structure (which is around 350 bytes on
a 64 bit machine). Is this right? I could understand if it read:
skb->truesize = size + sizeof(struct skb_shared_info);
or even:
skb->truesize = size + sizeof(struct sk_buff) + sizeof(struct
skb_shared_info);
What would be ideal is if the overhead was hidden from the user, so that
a 64K buffer held 64K of data, but that might be asking for too much.
Does anyone have an explaination/history on this? I would like to
propose enhancing/changing it.
Thanks,
Charlie Brett / HP
-
: send the line "unsubscribe linux-net" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
