Re: [PATCH] Speedup recursive by flushing index only once for all entries

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Linus Torvalds <torvalds@xxxxxxxx> writes:

> On Thu, 11 Jan 2007, Alex Riesen wrote:
>> 
>> It must have been large leak, as I really have seen the memory usage
>> dropping down significantly.
>
> I really think it was about 6MB (or whatever your index file size was) per 
> every single resolved file. I think merge-recursive used to flush the 
> index file every time it resolved something, and every flush would 
> basically leak the whole buffer used to write the index.

This does not change the conclusion "leaking is bad", but it was
not as bad as "the whole buffer used to write the index".

There are 10 4-byte ints, 20-bye SHA1, and a short plus pathname
and padding stored per a file and Alex has 44k files.  They are
not included in that *data the code was leaking, I would suspect
maybe a meg or so per path.

The version of merge-recursive in 'next' would not write out the
index at all until the very end once, so that makes this leak
somewhat irrelevant for that particular program ;-) but thanks
for the fix.

Here is the list of what I have queued for 'master' (I am
sending the list because it will be some time before I can push
them out):

Eric Wong (1):
      Avoid errors and warnings when attempting to do I/O on zero bytes

Junio C Hamano (5):
      Document git-init
      index-pack: write-or-die instead of unchecked write-in-full.
      config-set: check write-in-full returns in set_multivar
      git-rm: do not fail on already removed file.
      git-status: wording update to deal with deleted files.

Linus Torvalds (3):
      write-cache: do not leak the serialized cache-tree data.
      write_in_full: really write in full or return error on disk full.
      Better error messages for corrupt databases



-
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]