Last week when we were talking about spawning rdiff-backup to backup our systems, we diverged into discussing app/apache logs and the somewhat complicated system we currently have for grabbing those logs. Right now we have a list of hosts on log02 that it should grab logs from. Those hosts need to have rsyncd running on them to allow access from log02 to fetch the /var/log/httpd/ path from them. That requires 2 things to be coupled and it is a bit awkward if you set up a host that is tricky to access from log02 or isn't on the vpn. In general I also am not in love with having to have rsyncd listening on systems - even if it is ip-restricted. So the thought was we could do something like this on log02: 1. setup an ssh key on log02 that can run rsync to /var/log/httpd on all hosts 2. make any host that needs to have its logs retrieved be marked in the ansible inventory host/group vars 3. git clone public-ansible-repo onto log02 4. use group_by to construct a group of the hosts which can then be retrieved using rsync. The sole reason for using ansible here is so we can keep the log sync info in our inventory and to parallelize the retrieval of logs. This is more or less identical to what we talked about for backups using rdiff-backup. When we were discussing this Luke mentioned then using tbgrep(https://pypi.python.org/pypi/tbgrep) to search the resulting files and compile a set of tracebacks our apps are dumping out. If we have all the logs on log02 generating a report like this would be pleasantly kept away from the rest of our hosts and could give us reasonably useful reports of brokenness. I'd love some feed back on if this is all crazy or not :) -sv _______________________________________________ infrastructure mailing list infrastructure@xxxxxxxxxxxxxxxxxxxxxxx https://admin.fedoraproject.org/mailman/listinfo/infrastructure
