Re: Git and Very Large Codebases

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 04/20/2010 11:00 AM, Brian Herman wrote:
> Can git support codebases on the order of magnitude of Gigabytes?
> For example, a game developer might want to use git as their SCM
> software but their codebases can be in gigabytes in scale.
> If git cannot support this how would one begin to create this support?
> 

It depends more on the nature of the files than the size of the total
code. With large binary files and such, git will not perform very well.
On a code-tree of 1-30KiB source-code files it will perform
fantastically (compared to alternatives, that is; Nothing in the entire
world will handle multiple gigabytes faster than it can read it from
the disk).

I'd suggest splitting the repository in several different ones to
make git handle it well. Movies go in one repository, audio go in
another, pictures and whatnot in a third, and then you tie it all
together with a super-repo that holds the code and the build-system.

In repositories storing mainly binary files you go through the
gitattributes(5) man-page and set the necessary attributes for
the files in question. In your case, I'd suspect the .gitattributes
file would look something like this:

*.avi -diff -delta -crlf -merge
*.jpg -diff -delta -crlf -merge
*.mp3 -diff -delta -crlf -merge

This will prevent git from attempting delta-compression on the
files. It will also not try to create diffs from them, or perform
crlf-conversion or merge them. The gitattributes(5) man page has
more information regarding this.

Some trial and error with a small test-repository will help you
sort this out and understand what it means.

-- 
Andreas Ericsson                   andreas.ericsson@xxxxxx
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Considering the successes of the wars on alcohol, poverty, drugs and
terror, I think we should give some serious thought to declaring war
on peace.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]