Search squid archive

Re: Frequent cache rebuilding

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Andreev Nikita wrote:
Hi.

What are the reasons of squid rebuilding it's cache? Sometimes after
startup sometimes during work it starts to rebuild cache. I see it in
cahce.log:

Squid rebuilds the swap.state file when a logrotate (and I think reconfigure) is called for. Usually it's not a big deal. My main cache has about 13 million objects and rebuilding the cache takes just over 10 seconds:

2009/01/15 04:02:27| storeDirWriteCleanLogs: Starting...
...
2009/01/15 04:02:39|   Finished.  Wrote 12918887 entries.
2009/01/15 04:02:39|   Took 12.4 seconds (1044380.1 entries/sec).

and

2009/01/16 04:02:29| storeDirWriteCleanLogs: Starting...
...
2009/01/16 04:02:40|   Finished.  Wrote 12906114 entries.
2009/01/16 04:02:40|   Took 11.3 seconds (1138371.9 entries/sec).

2009/01/16 09:30:24| Starting Squid Cache version 2.6.STABLE6 for x86_64-redhat-linux-gnu...
2009/01/16 09:30:24| Process ID 2625
2009/01/16 09:30:24| With 1024 file descriptors available
2009/01/16 09:30:24| Using epoll for the IO loop
2009/01/16 09:30:24| DNS Socket created at 0.0.0.0, port 60302, FD 5
2009/01/16 09:30:24| Adding domain mydomain.com from /etc/resolv.conf
2009/01/16 09:30:24| Adding nameserver 22.179.1.11 from /etc/resolv.conf
2009/01/16 09:30:24| Adding nameserver 22.179.1.15 from /etc/resolv.conf
2009/01/16 09:30:24| User-Agent logging is disabled.
2009/01/16 09:30:24| Referer logging is disabled.
2009/01/16 09:30:24| Unlinkd pipe opened on FD 10
2009/01/16 09:30:24| Swap maxSize 16777216 KB, estimated 1290555 objects
2009/01/16 09:30:24| Target number of buckets: 64527
2009/01/16 09:30:24| Using 65536 Store buckets
2009/01/16 09:30:24| Max Mem  size: 393216 KB
2009/01/16 09:30:24| Max Swap size: 16777216 KB
2009/01/16 09:30:24| Local cache digest enabled; rebuild/rewrite every 3600/3600 sec
2009/01/16 09:30:24| Store logging disabled
2009/01/16 09:30:24| Rebuilding storage in /var/spool/squid (CLEAN)
2009/01/16 09:30:24| Using Least Load store dir selection
2009/01/16 09:30:24| Set Current Directory to /var/spool/squid
2009/01/16 09:30:24| Loaded Icons.
2009/01/16 09:30:26| Accepting proxy HTTP connections at 0.0.0.0, port 8080, FD 11.
2009/01/16 09:30:26| Accepting ICP messages at 0.0.0.0, port 3130, FD 12.
2009/01/16 09:30:26| WCCP Disabled.
2009/01/16 09:30:26| Ready to serve requests.
2009/01/16 09:30:35| Store rebuilding is  0.4% complete
2009/01/16 09:30:53| Store rebuilding is  1.1% complete
2009/01/16 09:31:08| Store rebuilding is  1.5% complete
2009/01/16 09:31:24| Store rebuilding is  2.3% complete
2009/01/16 09:31:44| Store rebuilding is  3.1% complete
2009/01/16 09:32:01| Store rebuilding is  3.4% complete
...
2009/01/16 10:32:41| Store rebuilding is 98.1% complete
2009/01/16 10:33:06| Store rebuilding is 98.5% complete
2009/01/16 10:33:26| Store rebuilding is 98.9% complete
2009/01/16 10:33:53| Store rebuilding is 99.6% complete
2009/01/16 10:34:13| Done reading /var/spool/squid swaplog (1072895 entries)
2009/01/16 10:34:13| Finished rebuilding storage from disk.
2009/01/16 10:34:13|   1072895 Entries scanned
2009/01/16 10:34:13|         0 Invalid entries.
2009/01/16 10:34:13|         0 With invalid flags.
2009/01/16 10:34:13|   1044791 Objects loaded.
2009/01/16 10:34:13|         0 Objects expired.
2009/01/16 10:34:13|         0 Objects cancelled.
2009/01/16 10:34:13|      7253 Duplicate URLs purged.
2009/01/16 10:34:13|     20851 Swapfile clashes avoided.
2009/01/16 10:34:13|   Took 3829.6 seconds ( 272.8 objects/sec).

Yikes.  That's slow...

2009/01/16 10:34:13| Beginning Validation Procedure
2009/01/16 10:35:04|    262144 Entries Validated so far.
2009/01/16 10:35:17|    524288 Entries Validated so far.
2009/01/16 10:35:25|    786432 Entries Validated so far.
2009/01/16 10:35:43|   Completed Validation Procedure
2009/01/16 10:35:43|   Validated 1044747 Entries
2009/01/16 10:35:43|   store_swap_size = 15131260k
2009/01/16 10:37:58| storeLateRelease: released 4184 objects

I never stop squid abnormally and I never restart server abnormally.
So I don't understand why does squid do it. It takes a lot of CPU
resources and about an hour of time.

What kind of CPU resources?  Userspace, system, or wait?

It never happened before. But now it happens several times a day.

Could somebody help whith that?

P.S. I have 18G partition with 16G granted for squid cache and 2G free
for temporary squid files. So I hope it's enough space for squid to
perform it's everyday tasks.

Ah. A lot of file systems start to perform poorly as they approach capacity. I'd suggest setting your cache_dir to 13G and let squid clear out the excess or grow the partition to 30G (if you have the space).

Regards,
LPIC-1, EMCPA
Nikita Andreev

Chris


[Index of Archives]     [Linux Audio Users]     [Samba]     [Big List of Linux Books]     [Linux USB]     [Yosemite News]

  Powered by Linux