Nguyen Thai Ngoc Duy <pclouds@xxxxxxxxx> wrote: > Naah, git pack-objects needs list of commit tips. Try > git for-each-ref|cut -c 1-40|git pack-objects --all --stdout > /dev/null Jared Hance <jaredhance@xxxxxxxxx> wrote: > I would look in the code for malloc calls that don't have a free call, > or spots where free calls might not be hit. Hello Jared and Nguyen, Thank you Nguyen for your command. I can now reproduce the problem without needing the network. I have been following Jared lead today on a potential memory leak. Here is what I found out. I downloaded the latest release of git 1.7.2.1 and compiled it with debugging support. I ran valgrind on the command and found two memory leaks. I put the output at the bottom of the e-mail as it's not very interesting. I patched one of the leak in pack_objects.c but got the same problem: over 4G of memory consumption for a 4G repository. I've come to the conclusion that it's not a memory leak. This afternoon I put macro around the following functions: xmalloc xmallocz, xrealloc, xcalloc and xmmap. It reported the line of code and size passed in each functions. I then run the result through a script that totaled the amount used by each bit of code. Here are the top 3 consumers: | function | source | size in M | |----------+----------------------------+-----------| | xrealloc | builtin/pack-objects.c:690 | 86 | | xmallocz | patch-delta.c:36 | 301 | | xmmap | sha1_file.c:772 | 4393 | I expected the malloc to take 4G but was surprised it didn't. It seems to be mmap taking all the memory. I am not familiar with that function, it looks like it's mapping memory to a file... Is it reasonable to mmap so much memory? Today I chatted with someone on freenode #git and he reported the same problem on his 2G repository, I am glad I am not the only one seeing this ;) I tried reading the code but it's going over my head. I'll look at is some more next monday. If anyone is familiar with the code source of git I would love to have some insight into this. Take care, Ivan Kanis PS: output of valgrind --leak-check=full 65 bytes in 1 blocks are definitely lost in loss record 4 of 7 at 0x4C2260E: malloc (vg_replace_malloc.c:207) by 0x4C22797: realloc (vg_replace_malloc.c:429) by 0x4C600D: xrealloc (wrapper.c:80) by 0x4B7939: strbuf_grow (strbuf.c:70) by 0x4B80BA: strbuf_addf (strbuf.c:201) by 0x4832EF: system_path (exec_cmd.c:37) by 0x483411: setup_path (exec_cmd.c:104) by 0x404AF2: main (git.c:536) 512 bytes in 1 blocks are definitely lost in loss record 5 of 8 at 0x4C203E4: calloc (vg_replace_malloc.c:397) by 0x4C5F9D: xcalloc (wrapper.c:96) by 0x445741: cmd_pack_objects (pack-objects.c:2117) by 0x4048EE: handle_internal_command (git.c:270) by 0x404B03: main (git.c:470) -- http://kanis.fr Everything should be made as simple as possible, but not simpler. -- Albert Einstein -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html