On Thursday 02 June 2005 02:13, the author Fredrik Steen contributed to the dialogue on- [users@httpd] Apache 2.x remote logging: >I'm in the process of building a web farm consisting of 10-20-30 something > web servers handling 300k of virtual hosts and have started to look at how > to handle logging for these hosts. > >Do anyone have any good ideas how to handle that massive amount of log > files? Preferably using some sort of remote log collector. > >I have looked at: > - mod-witch: http://savannah.nongnu.org/projects/mod-witch (Will syslog > handle it?) - mod_log_spread: http://www.backhand.org/mod_log_spread/ > (Apache 2.x module status?) > - mod_log_sql: http://www.outoforder.cc/projects/apache/mod_log_sql/ > (Performance?) > >Any recommendations or ideas to handle this? With such a massive logging requirement I feel some analysis of the purpose to you intend to fulfill by using the logs could well drive the selection of methods & processes for dealing with them and hence and your choice of solution. Are there any top level plans which describes each purpose and some idea of data volumes applicable for each purpose you feel able to share with us? To give you an example of why I ask this question some of your logging data may simply need to be analyzed and the raw data dumped. For such a purpose you have a choice of either local analysis and cyclical dumping of raw data with forwarding of processed results to a central collector or, in the alternative, raw data forwarding and central or departmental analysis. The choice of technique may depend upon the usage point as well as considerations of band width, processing and storage. (For example do you need to have a centralized record of every http access to each of your virtual hosts files - or is that data only needed by the virtual host but you may need a centralized summary for statistical and marketing purposes). The sharing of a plan would help oil observations. David -- 40 yrs navigating and computing in blue waters. English Owner & Captain of British Registered 60' bluewater Ketch S/V Taurus. Currently in San Diego, CA. Sailing May/June bound for Europe via Panama Canal. --------------------------------------------------------------------- The official User-To-User support forum of the Apache HTTP Server Project. See <URL:http://httpd.apache.org/userslist.html> for more info. To unsubscribe, e-mail: users-unsubscribe@xxxxxxxxxxxxxxxx " from the digest: users-digest-unsubscribe@xxxxxxxxxxxxxxxx For additional commands, e-mail: users-help@xxxxxxxxxxxxxxxx
|