On Tue, Apr 3, 2012 at 12:55 PM, Martin Fick <mfick@xxxxxxxxxxxxxx> wrote: > Nguyen Thai Ngoc Duy <pclouds@xxxxxxxxx> wrote: >>> we could even store the data in a separate file to >>> retain indexv2 compatibility). >>> >>> So it's sort-of a cache, in that it's redundant with the actual data. >>> But staleness and writing issues are a lot simpler, since it only >>gets >>> updated when we index the pack (and the pack index in general is a >>> similar concept; we are "caching" the location of the object in the >>> packfile, rather than doing a linear search to look it up each time). >> >>I think I have something like that, (generate a machine-friendly >>commit cache per pack, staying in $GIT_DIR/objects/pack/ too). It's >>separate cache staying in $GIT_DIR/objects/pack, just like pack-.idx >>files. It does improve rev-list time, but I'd rather wait for packv4, >>or at least be sure that packv4 will not come anytime soon, before >>pushing the cache route. > > I would love to try those patches out if you have them? There you go. Note that these patches are not of high quality. I did not even run "make test". To create commit cache, simply run index-pack, e.g. $ git repack -ad $ git index-pack --stdin < .git/objects/pack/pack-XXX.pack It will create two more files, pack-XXX.sha1 and pack-XXX.sidx. On linux-2.6.git, "git rev-list --all --quiet HEAD" takes 1.9s with the patches and 6.6s without. Disk usage: total 531M 56M pack-ab843186bdfb00956c1b1c0cdb4ed5e4aa3e549e.idx 460M pack-ab843185bdfb00956c1b1c0cdb4ed5e4aa3e549e.pack 9.7M pack-ab843185bdfb00956c1b1c0cdb4ed5e4aa3e549e.sha1 5.3M pack-ab843185bdfb00956c1b1c0cdb4ed5e4aa3e549e.sidx Nguyễn Thái Ngọc Duy (3): parse_commit_buffer: rename a confusing variable name Add commit cache to help speed up commit traversal Add parse_commit_for_rev() to take advantage of sha1-cache Makefile | 3 + builtin/index-pack.c | 113 ++++++++++++++++++++++++++++++++++- builtin/reflog.c | 2 +- cache.h | 9 +++ commit.c | 46 +++++++++++--- commit.h | 1 + log-tree.c | 2 +- pack-write.c | 11 +++- pack.h | 1 + revision.c | 10 ++-- sha1_cache.c | 161 ++++++++++++++++++++++++++++++++++++++++++++++++++ sha1_cache.h | 6 ++ sha1_file.c | 12 ++++- test-sha1-cache.c | 19 ++++++ upload-pack.c | 2 +- walker.c | 2 +- 16 files changed, 377 insertions(+), 23 deletions(-) create mode 100644 sha1_cache.c create mode 100644 sha1_cache.h create mode 100644 test-sha1-cache.c -- 1.7.3.1.256.g2539c.dirty -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html