Amos referred to a mail thread where the issues of the 32K limit were discussed.
This thread does not suggest that 32K is a hard shared memory system limit
but merely that 32K is a Squid-specific design limitation.
Oracle and other RDBMS use very large shared memory segments of
multiple gigabytes. 32MB for SHMMAX is a default on some
systems but can be altered and set much higher.
The implication of the design decisions
- use one page of 32K for shared objects
- use multi-process architecture vs multi-thread architecture with a shared memory segment
imply the current limit that the maximum size for a shared object
is 32K and that larger objects cannot be shared between worker processes
using the shared memory segment. I think that everybody involved
does not like the limitation. Observing the conversations
in the squid-dev list it seems there is not sufficient time/funding to
do everything at this time.
Marcus
On 08/05/2012 07:37 PM, Linda W wrote:
Linda W wrote:
Amos Jeffries wrote:
We are still limited to one page,
---
1 page or 1 segment/item?
----
I don't know who 'we' is... but on x86_64 linux, I was able to use the
perl SysV::IPC calls shmget/shmwrite/shmread/shmctl to allocate up to
my system's run-time limit (which I can up if needed) of 32MB/shm segment.
It looks like there is an underlying granularity of 8KB, but shouldn't
it be easy to simply use the shm interface and allocate exact size segments
to hold shared files?
Hmmm......
Either that or allocate in largest size chunksize available and sub-divide it.
As for disk -- if the index of files that were stored on disk -- why
couldn't the processes share a file cache? Certainly you don't want
two separate processes downloading the same file at the same time -- that
would really hurt bandwidth...
