On Thu, 26 Feb 2015 08:11:50 -0700 Kevin Fenzi <kevin@xxxxxxxxx> wrote: > We have been having some persistent issues with kojipkgs01 lately. > > kojipkgs01 is our squid proxy in front of koji builds. It allows users > and builders to get fast access to packages. (When it's working). > > Lately, it's been working fine at first, then in a few days or so it > starts getting really slow. Downloads go from 25M/s to 200k/sec and > sometimes things even just timeout. > > Restarting squid seems to fix this... for a few more days. > There is never any errors on the box, i/o, load and everything is > fine. > > I looked this morning a bunch at options and adjusted the memory cache > down in case we were hitting some kind of issue with memory cache. > > I'd like +1's for that change, and also to solicit ideas for what we > can do to fix this once and for all (if these changes don't do so). So, I finally tracked down the issues with smp mode on squid. ;) It's two slightly different bugs on Fedora and RHEL. Fedora: https://bugzilla.redhat.com/show_bug.cgi?id=1176318 RHEL: https://bugzilla.redhat.com/show_bug.cgi?id=1102842 In Fedora squid is set to use a /var/run/squid dir for pid/ipc state, but it doesn't have a tmpfiles.d config to make that dir on boot, so it only works until you reboot. In RHEL, they compiled it to use /var/run/squid.pid for the pid file, and the ipc state files can't be written to /var/run/ as squid user, so they nicely silently fail to talk to each other and no one binds to the port and nothing works. We can work around this issue for now on RHEL with: * Set pid_filename /var/run/squid/squid.pid in the squid.conf. * Add a selinux local policy or set permissive mode: #============= squid_t ============== allow squid_t var_run_t:sock_file { read write create }; * Add a /etc/tmpfiles.d/squid.conf with: D /var/run/squid 0755 squid squid - So, I guess we can wait and see if the slowdown happens again, and if so, try the above to enable smp mode and see if it helps any? kevin
Attachment:
pgph4cbnyqZor.pgp
Description: OpenPGP digital signature
_______________________________________________ infrastructure mailing list infrastructure@xxxxxxxxxxxxxxxxxxxxxxx https://admin.fedoraproject.org/mailman/listinfo/infrastructure
