On Thu, 21 Jan 2010, Linus Torvalds wrote:
>
> Sure. Except, as I mentioned, it's not just git-hash-object. It's _all_ of
> them.
>
> The total space savings wasn't 1.7M, it was 12M.
There are some other interesting cases. For example, look at this:
[torvalds@nehalem git]$ size show-index.o git-show-index
text data bss dec hex filename
1310 0 1024 2334 91e show-index.o
222706 2296 112720 337722 5273a git-show-index
ie a trivial program like 'show-index' has ballooned to 220kB. Let's look
at why:
[torvalds@nehalem git]$ nm show-index.o | grep ' U '
U die
U fread
U free
U printf
U sha1_to_hex
U stdin
U usage
U xmalloc
ok, if you ignore standard library things (which will be from a shared
library anyway), it really only wants totally trivial things: die,
xmalloc, and sha1_to_hex. Those should be a few hundred bytes, not a few
hundred _kilo_bytes.
So what happens?
- sha1_to_hex brings in all of sha1_file.c, even though it doesn't need
any of it. Ok, that's easily fixed: split up the hex helpers into a
file of its own ("hex.c")
- "die()" brings in usage.c, which is actually designed correctly, so it
is all fine. No extra pain there. Sure, we'll get some other trivial
stuff from there, but we're talking maybe a kilobyte of code.
- "xmalloc()" brings in the trivial wrappers.
OOPS.
Those wrappers bring in zlib (through git_inflate*), which is not a huge
issue, we coult just move the git_inflate*() wrappers to its own file.
Trivial. But the wrappers also bring in:
- xmalloc/xrealloc/xstrdup:
U release_pack_memory
which in turn brings in _all_ of the rest of the git libraries. End
result: a trivial git helper program that _should_ be a couple of
kilobytes in size ends up being 200+kB of text, and 900kB with debug
information.
Absolutely _none_ of which is in the least useful.
Oh well.
We could fix it a few ways
- ignore it. Most git programs will get the pack handling functions
anyway, since they want to get object reading.
- as mentioned, just build in _everything_ so that we only ever have one
binary
- get rid of release_pack_memory() entirely. We have better ways to limit
pack memory use these days, but they do require configuration (we do
have a default packed_git_limit, though, so even without any explicit
configuration it's not insane).
- don't have explicit knowledge about 'release_pack_memory' in xmalloc,
but instead have the packing functions register a "xmalloc
pressure_reliever function". So then programs that have pack handling
will register the fixup function, and programs that don't will never
even know.
Hmm? We have about 20 external programs that may hide issues like this.
Linus
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html