On 2/14/2020 1:22 PM, Jeff King wrote: > We can easily support BLOB_NONE filters with bitmaps. Since we know the > types of all of the objects, we just need to clear the result bits of > any blobs. > > Note two subtleties in the implementation (which I also called out in > comments): > > - we have to include any blobs that were specifically asked for (and > not reached through graph traversal) to match the non-bitmap version I have a concern here, but maybe I'm worrying about nothing. When a partial clone asks for a pack of missing blobs, will your code create an empty bitmap and then add bits to that bitmap one-by-one instead of appending to a simple object list? In the typical case where we ask for specific commits and trees, we expect a very small number of blobs to add to the resulting bitmap. When no commits or trees are included in the wants, then we don't need the bitmap at all. IIRC an EWAH bitmap is relatively expensive to update bits one at a time, so this is not incredibly efficient. I apologize that I don't have the time to dig into this myself. > - we have to handle in-pack and "ext_index" objects separately. > Arguably prepare_bitmap_walk() could be adding these ext_index > objects to the type bitmaps. But it doesn't for now, so let's match > the rest of the bitmap code here (it probably wouldn't be an > efficiency improvement to do so since the cost of extending those > bitmaps is about the same as our loop here, but it might make the > code a bit simpler). > > Here are perf results for the new test on git.git: > > Test HEAD^ HEAD > -------------------------------------------------------------------------------- > 5310.9: rev-list count with blob:none 1.67(1.62+0.05) 0.22(0.21+0.02) -86.8% ... > diff --git a/t/perf/p5310-pack-bitmaps.sh b/t/perf/p5310-pack-bitmaps.sh > index e52f66ec9e..936742314c 100755 > --- a/t/perf/p5310-pack-bitmaps.sh > +++ b/t/perf/p5310-pack-bitmaps.sh > @@ -47,6 +47,11 @@ test_perf 'rev-list (objects)' ' > git rev-list --all --use-bitmap-index --objects >/dev/null > ' > > +test_perf 'rev-list count with blob:none' ' > + git rev-list --use-bitmap-index --count --objects --all \ > + --filter=blob:none >/dev/null > +' I wondered why you chose to extend these tests instead of using p5600-partial-clone.sh, but I guess this script definitely creates the bitmap for the test. When I tested p5600-partial-clone.sh below, I manually repacked the Linux repo to have a bitmap: Test v2.25.0 HEAD ---------------------------------------------------------------------------- 5600.2: clone without blobs 79.81(111.34+11.35) 36.00(69.37+7.30) -54.9% 5600.3: checkout of result 45.56(114.59+4.81) 46.43(80.50+5.41) +1.9% Perhaps results for these tests would also be appropriate for your commit messages? Note the +1.9% for the checkout. It's unlikely that this is actually something meaningful, but it _could_ be related to my concerns above about building a blob list from an empty bitmap. Thanks, -Stolee
