Somebody complained to me off-list recently that for-each-ref was slow
when doing a trivial listing (like refname + objectname) of a large
number of refs, even though you could get mostly the same output by
just dumping the packed-refs file.
So I was nerd-sniped into making this horrible series, which does speed
up some cases. It was a combination of "how fast can we easily get it",
plus I was curious _which_ parts of for-each-ref were actually slow.
In this version there are 2 patches, tested against 'git for-each-ref
--format="%(objectname) %(refname)"' on a fully packed repo with 500k
refs:
- directly outputting items rather than building up a ref_array yields
about a 1.5x speedup
- avoiding the whole atom-formatting system yields a 2.5x speedup on
top of that
I had originally written it against a slightly older version of Git,
before show_ref_array_item() was inlined. In that version, there was a
middle ground where we still created a ref_array_item for each ref, and
then fed it to the "quick" formatter (so we could see the cost of
having to malloc/free a bunch of ref_array_item structs). The numbers
there wre:
- streaming out items gave a 1.5x speedup
- using the "quick" formatter gave a 1.8x speedup
- avoiding the extra malloc/free for each item gave a 1.4x
which makes sense; 1.8 * 1.4 is ~2.5, so it's the same speedup just
broken down further.
I'm not sure I'm really advocating that we should do something like
this, though it is tempting because it does make some common queries
much faster. But I wanted to share it here to give some insight on where
the time may be going in ref-filter / for-each-ref.
[1/2]: ref-filter: hacky "streaming" mode
[2/2]: ref-filter: implement "quick" formats
builtin/for-each-ref.c | 7 +++
ref-filter.c | 113 +++++++++++++++++++++++++++++++++++++----
ref-filter.h | 21 ++++++++
3 files changed, 132 insertions(+), 9 deletions(-)
-Peff
