Hi Tek,
I've had to make several modifications to the standard setup to get it to handle the actual requests coming in, the cache (without disks) is able to maintain around 1800RPS now - of course I don't expect the disks to ever get that high.
I'm running 4.11, the relevant kernel tweaks are --
options SMP
options APIC_IO
options MSGMNB=32768
options MSGMNI=160
options MSGSEG=2048
options MSGSSZ=256
options MSGTQL=8192
options MAXDSIZ="(1536*1024*1024)"
options DFLDSIZ="(1536*1024*1024)"
maxusers 1024
In loader.conf --
kern.ipc.nmbclusters="32768"
That leaves me with a max process limit of 1.5gig, enough memory for diskd and a netstat -m like so:
258/1040/131072 mbufs in use (current/peak/max):
258 mbufs allocated to data
256/1018/32768 mbuf clusters in use (current/peak/max)
As for sysctl tunables --
vfs.vmiodirenable=1
kern.ipc.maxsockbuf=2097152
kern.ipc.somaxconn=8192
kern.ipc.maxsockets=16424
kern.maxfiles=65536
kern.maxfilesperproc=32768
net.inet.tcp.rfc1323=1
net.inet.tcp.delayed_ack=0
net.inet.tcp.sendspace=32768
net.inet.tcp.recvspace=65535
net.inet.ip.portrange.last=44999
net.inet.ip.portrange.hifirst=45000
net.inet.tcp.keepidle=15000
net.inet.tcp.keepintvl=5000
net.inet.tcp.keepinit=60000
net.inet.tcp.msl=6000
To sum up the above, I have increase my maxfiles, changed the send/receive space and increased the available ports to squid. I've also modified the timeout and msl settings for tcp to get it to drop FIN_WAIT TIME_WAIT etc sessions which are wasting ports.
I'm almost certain the diskd crash is an actual crash and not a slow down from my experience..
Thanks
Dave
-----Original Message-----
From: Tek Bahadur Limbu [mailto:teklimbu@xxxxxxxxxxxx]
Sent: Wednesday, November 14, 2007 1:48 PM
To: Dave Raven
Cc: 'John Moylan'; 'squid-users'
Subject: Re: Squid Performance (with Polygraph)
Hi Dave,
Dave Raven wrote:
> I have seen the error messages before, but not during these tests. diskd definitely seems to delay the time-till-crash by a lot - as I understand it the problems in diskd are crashes under high load, not that it slows it down right?
From my experience, YES, DISKD crashes under high load but does not
actually slows Squid down. It slows Squid initially while rebuilding
it's cache after the crash but recovers quite fast not to hamper
performance.
Only under certain circumstances, will it cause the cache to go beyond
repair and the only way out is to wipe out the cache and rebuild it from
scratch.
The time for the DISKD crashes also seems to vary alot from crashing
multiple times a day to a single crash a week or two.
From your earlier posts, since all your testings lasted from 10 minutes
to 18 hours, maybe the DISKD crash did not appear during that time.
Also your FreeBSD version 4.x might have also made the difference!
Can you post your FreeBSD 4.x KERNEL parameters that you compiled for
your testing purposes?
Thanking you...
>
> Thanks for the help
> Dave
>
> -----Original Message-----
> From: redsnapper8t8@xxxxxxxxx [mailto:redsnapper8t8@xxxxxxxxx] On Behalf Of John Moylan
> Sent: Wednesday, November 14, 2007 12:39 PM
> To: Dave Raven
> Subject: Re: Squid Performance (with Polygraph)
>
> Doesn't diskd have a bug whereby it has issues under heavy load.
> http://www.squid-cache.org/bugs/show_bug.cgi?id=761 . If so, I am
> surprised that it is behaving best under heavy load.
> http://www.squid-cache.org/Versions/v2/2.6/squid-2.6.STABLE16-RELEASENOTES.html
>
> J
>
>
>
>
--
With best regards and good wishes,
Yours sincerely,
Tek Bahadur Limbu
System Administrator
(TAG/TDG Group)
Jwl Systems Department
Worldlink Communications Pvt. Ltd.
Jawalakhel, Nepal
http://www.wlink.com.np
http://teklimbu.wordpress.com
