On Sat, Mar 15, 2008 at 02:44:42PM -0700, Jakub Narebski wrote: > Petr Baudis <pasky@xxxxxxx> writes: > > > On repo.or.cz (permanently I/O overloaded and hosting 1050 project + > > forks), > > It looks like repo.or.cz is overwhelmed by its success. I hope that > now that there are other software hosting sites with git hosting > (Savannah, GitHub, Gitorious,...) the number of projects wouldn't grow > as rapidly. Actually, it was overwhelmed to so much by its success but by lack of good maintenance. ;-) I gave it some love again for the past week and the improvement was, well, overwhelming. :-) I finally fixed tons of failures and broken repositories, and most importantly repacked some of the big repositories with object databases in pretty horrid shape. The effect has been immense, having everything in database of 1/3 the size and single big pack drastically reduced the I/O load. Scenario: Site with about 1100 repositories weighting 13GB, running a fetch job for about 200 of them hourly. About two git-daemon requests per minute and 10 gitweb requests per minute (the last two numbers are taken quite sloppily over a small sample of the last ten minutes ;-). Site is running on 2x1GHz P3 with 2G RAM, repository is on hw RAID5. (We are currently preparing to migrate it to a more powerful machine.) Before, the load on the server would be normally about 6 to 15 _all the time_ and bunch of git-related processes would be permanently eating some CPU and crunch on the disk. After introducing the index caching and repacking the repositories, the load seems to be around 1 at most and hardly seems to come above 3; all feels very snappy. So for anyone running a hosting site, make sure your repositories are nicely packed. It makes huge difference to the I/O load! > Another solution would be to divide projects list page into pages, > perhaps adding search box for searching for a project (by name, by > description and by owner). > > Nevertheless even with pagination, if we want to have "sort by last > update" we do need caching. Yes, I'm pondering about pagination, but because of web clients, not the server load; it takes firefox on my notebook noticeable time to render this list already, and it's rather big too. Ideas are welcome here. My current plan is to have a [Search project] box at the front page, together with direct link to 'show all'. Other than that, what makes sense to display on the front page? I think recently added projects (age < 1 week) for sure. I'm not so sure about recently changed projects - maybe it is better to keep the front page cruft-free. -- Petr "Pasky" Baudis Whatever you can do, or dream you can, begin it. Boldness has genius, power, and magic in it. -- J. W. von Goethe -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
