On 10/03/2017 16:15, Vegard Nossum wrote:
I've used AFL to generate a corpus of pack files that maximises the edge
coverage for 'git index-pack'.
This is a supplement to (and not a replacement for) the regular test cases
where we know exactly what each test is checking for. These testcases are
more useful for avoiding regressions in edge cases or as a starting point
for future fuzzing efforts.
To see the output of running 'git index-pack' on each file, you can do
something like this:
make -C t GIT_TEST_OPTS="--run=34 --verbose" t5300-pack-object.sh
I observe the following coverage changes (for t5300 only):
path old% new% pp
----------------------------------------
builtin/index-pack.c 74.3 76.6 2.3
pack-write.c 79.8 80.4 .6
patch-delta.c 67.4 81.4 14.0
usage.c 26.6 35.5 8.9
wrapper.c 42.0 46.1 4.1
zlib.c 58.7 64.1 5.4
And if you add this simple patch on top (sorry, I didn't think of it
until after I'd sent the previous e-mail):
diff --git a/t/t5300-pack-object.sh b/t/t5300-pack-object.sh
index 19e02ffc2..db705ba5c 100755
--- a/t/t5300-pack-object.sh
+++ b/t/t5300-pack-object.sh
@@ -425,8 +425,10 @@ test_expect_success 'index-pack <pack> works in
non-repo' '
test_expect_success 'index-pack edge coverage' '
for pack in "$TEST_DIRECTORY"/t5300/*.pack
do
- rm -rf "${pack%.pack}.idx" &&
- test_might_fail git index-pack $pack
+ rm -rf "${pack%.pack}.idx" tmp.pack tmp.idx &&
+ test_might_fail git index-pack $pack &&
+ test_might_fail git index-pack --strict $pack &&
+ test_might_fail git index-pack --stdin --fix-thin
tmp.pack < $pack
done
'
you get this change to the coverage profile instead:
path old% new% pp
----------------------------------------
alloc.c 58.1 67.4 9.3
builtin/index-pack.c 74.3 80.7 6.4
commit.c 13.9 17.4 3.5
date.c 3.5 4.2 .7
fsck.c 15.7 33.7 18.0
object.c 56.0 58.7 2.7
pack-write.c 79.8 81.4 1.6
patch-delta.c 67.4 81.4 14.0
path.c 31.6 32.1 .5
sha1_file.c 48.9 49.6 .7
tag.c 3.7 16.8 13.1
tree.c 36.6 37.5 .9
usage.c 26.6 35.5 8.9
wrapper.c 42.0 46.1 4.1
zlib.c 58.7 64.1 5.4
Of course, it's likely some of those gains can be found in other
testcases outside t5300 -- also, coverage isn't everything. Still seems
like a nice gain with very little effort.
Vegard