Re: Big project, slow access!

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Fri, 18 Sep 2009, Toan Pham wrote:

> Hi,
> 
> I use git to maintain a project that is at least 8 gigs in size.
> The project is a Linux from Scratch repository that includes source
> codes to approximately 2000 open source projects,
> gcc tool-chain, 1000+ configurations for different software packages,
> source code for different kernel versions,
> and many linux distributions/flavors resulted from this LFS build environment.
> 
> The git's object repository is now 4.6 gigs and consists of approx.
> 610,000 files and folders.
> The speed of git is now terribly slow.  Each time I use basic commands
> like 'git status' or 'git diff',
> it would take at least 5 minutes for git to give me back a result.
> Again, the machine that i run git on is a P4 3.2 gig-hertz with HT.

Howdy Toan, we have a similarly large repository ~405k files, the .git
folder fully packed is ~6GB. 

The advise to fully-pack your repository is likely going to have the
greatest impact on your performance in the short term, in the long term
however you might want to consider using git-filter-branch(1) or other
tools available to separate our the components of your current Git
reposotory into a series of repos.

The performance hit you're seeing likely has nothing to do with your
processor speed either, but rather your disk search speed (i'm waiting
for a new fancy SSD to help alleviate my issues ;))

> would  someone please recommend on how i can optimize git's performance?
> Git is so slow, are there better ways to manage a project like this?

Rethink how your project is laid out, and whether certain binaries files
need to sit in the tree, or can be build on a need-by-need basis.



Cheers
-R. Tyler Ballance
Slide, Inc.

Attachment: pgprx54xbYijS.pgp
Description: PGP signature


[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]