Re: git-repack: Outof memory

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Dongsheng Song <dongsheng.song@xxxxxxxxx> wrote:
> I'm import from subversion. The problem appears to be git-repack phase using too many memory:
> 
> $ git-repack -a -f -d --window=64 --depth=64
> Generating pack...
> Done counting 123497 objects.
> Deltifying 123497 objects.
>   24% (29677/123497) done
> 
> $ top
> 
>   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
>  3572 www-data  18   0 2591m 1.9g  528 R   13 94.8  81:48.98 git-pack-object

*ouch* You are probably running out of address space if you are on
a 32 bit architecture.  A 2.5 GiB virtual address space is pretty
close to the maximum allowed on most OSes.

The code that I'm sitting on but haven't yet completed rebasing
onto current Git would probably help here.

Do you have any existing .pack files in .git/objects/pack?  How big
are they and their corresponding .idx files?

git-repack will need to mmap every .pack and .idx in
.git/objects/pack, plus it needs working memory for each object
(123,497 of 'em) but as I recall its pretty frugal on its per-object
allocation.  It can easily work with as many as 2 million objects on
a 32 bit system, assuming the .pack and .idx files aren't too large.
 
-- 
Shawn.
-
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]