Search squid archive

Re: Caching large amounts of images...

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Forrest Aldrich wrote:

We are developing an infrastructure that will serve up images directly from a database. The numbers will vary into the thousands and higher.

In front of this process, I would like to place a cache of the most frequently accessed images, to reduce the load on the database server itself.

The images are small (under 200k).

My question is what is the most effective way to accomplish this (without a commercial cache solution like content distribution). Would Squid be appropriate. There are other approaches such as Apache's mod_cache_disk et al.

Squid would be appropriate. If your need is not too urgent, I would suggest deploying your test on Squid2.6 (which is still in the Release Candidate stage), as it apparently has improvements to the reverse proxy setup. Apache's mod_proxy or mod_cache_disk would certainly be another option. I'm not familiar enough with them to provide an opinion on which would be better.


Looking through the Squid book, there are a lot of tunables... including consideration of the underlying filesystem (the OS is open here, we can use Linux or FreeBSD (preferred)).

While the page at http://www.squid-cache.org/Doc/Users-Guide/opt/performance.html is old, the bullet points are still valid. The question of underlying file system is posed on occasion to the list. I've been subscribed to the list for close to two years now, and I have yet to see any compelling evidence of one over the other. Use what you are familiar with. Tune it to the best of your ability (at the very least follow the advice in the bullets from the above link. Squid is actively supported on (at least) Linux, FreeBSD and Windows. Pick your platform and go to town.


COSS sounds interesting, but it appears to be dead - or, at least, not ready for production. The web page I found hasn't been updated in a long time and indicated development being on Windows.

There has been some recent activity on COSS (check the mailing list archive), but it's not ready for production.


Any pointers would be appreciated (I'm new to caching), thank you.

In no particular order:
*Stay away from regex ACLs.
*Throw as much RAM as you can afford in to the caching servers.
*Use lots of disks.
* Don't RAID your cache disks.
*Stay away from the ufs store type (aufs seems to be the more stable than diskd, diskd might be a bit quicker). *Keep your cache_mem setting fairly low (it's only used for objects fetched from the network), let your system's disk caching work to your advantage.

Chris

[Index of Archives]     [Linux Audio Users]     [Samba]     [Big List of Linux Books]     [Linux USB]     [Yosemite News]

  Powered by Linux