Re: Leaving large binaries out of the packfile

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Paolo Bonzini <bonzini@xxxxxxx> wrote:
> On 06/10/2010 08:04 PM, Shawn O. Pearce wrote:
>> Joshua Jensen<jjensen@xxxxxxxxxxxxxxxxx>  wrote:
>>> Sometimes, 'git gc' runs out of memory.  I have to discover which file
>>> is causing the problem, so I can add it to .gitattributes with a
>>> '-delta' flag.  Mostly, though, the repacking takes forever, and I dread
>>> running the operation.
>>
>> If you have the list of big objects, you can put them into their
>> own pack file manually.  Feed their SHA-1 names on stdin to git
>> pack-objects, and save the resulting pack under .git/objects/pack.
>
> Do you know any simpler way than
>
> git log --pretty=format:%H | while read x; do
>   git ls-tree $x -- ChangeLog | awk '{print $3}'
> done | sort -u
>
> to do this?  I thought it would be nice to add --sha1-only to  
> git-ls-tree, but maybe I'm missing some other trick.

Maybe

  git rev-list --objects HEAD | grep ' ChangeLog'

pack-objects wants the output of rev-list --objects as input, file
name and all.  So its just a matter of selecting the right lines
from its output.

>> Assuming the pack was called pack-DEADC0FFEE.pack, create a file
>> called pack-DEADC0FFEE.keep in the same directory.  This will stop
>> Git from trying to repack the contents of that pack file.
>>
>> Now run `git gc` to remove those huge objects from the pack file
>> that contains all of the other stuff.
>
> That obviously wouldn't help if these large binaries are updated often,  
> however.

No, it doesn't.  But you still could do this on a periodic basis.
That way you only drag around a handful of recently created large
binaries during a typical `git gc`, and not the entire project's
history of them.

-- 
Shawn.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]