This is now based on 'master'. Did you know that 'fetch' and 'pull' read the index? I didn't, or this would have been an integration much earlier in the cycle. They read the index to look for the .gitmodules file in case there are submodules that need to be fetched. Since looking for a file by name is already protected, we only need to disable 'command_requires_full_index' and we are done. The 'ls-files' builtin is useful when debugging the index, and some scripts use it, too. We are not changing the default behavior which expands a sparse index in order to show all of the cached blobs. Instead, we add a '--sparse' option that allows us to see the sparse directory entries upon request. Combined with --debug, we can see a lot of index details, such as: $ git ls-files --debug --sparse LICENSE ctime: 1634910503:287405820 mtime: 1634910503:287405820 dev: 16777220 ino: 119325319 uid: 501 gid: 20 size: 1098 flags: 200000 README.md ctime: 1634910503:288090279 mtime: 1634910503:288090279 dev: 16777220 ino: 119325320 uid: 501 gid: 20 size: 934 flags: 200000 bin/index.js ctime: 1634910767:828434033 mtime: 1634910767:828434033 dev: 16777220 ino: 119325520 uid: 501 gid: 20 size: 7292 flags: 200000 examples/ ctime: 0:0 mtime: 0:0 dev: 0 ino: 0 uid: 0 gid: 0 size: 0 flags: 40004000 package.json ctime: 1634910503:288676330 mtime: 1634910503:288676330 dev: 16777220 ino: 119325321 uid: 501 gid: 20 size: 680 flags: 200000 (In this example, the 'examples/' directory is sparse.) Thanks! Updates in v2 ============= * Rebased onto latest ld/sparse-index-blame without issue. * Updated the test to use diff-of-diffs instead of a sequence of greps. * Added patches that remove the use of 'test-tool read-cache --table' and its implementation. Updates in v3 ============= * Fixed typo in commit message. * Added comments around doing strange things in an ls-files test. * Fixed adjacent typo in a test comment. Updates in v4 ============= * Rebased on to 'master' now that ld/sparse-index-blame is merged. * Change testing strategy to check exact output instead of using 'diff -u'. * Updated documentation to state that directories have a trailing slash. Derrick Stolee (5): fetch/pull: use the sparse index ls-files: add --sparse option t1092: replace 'read-cache --table' with 'ls-files --sparse' t1091/t3705: remove 'test-tool read-cache --table' test-read-cache: remove --table, --expand options Documentation/git-ls-files.txt | 5 + builtin/fetch.c | 2 + builtin/ls-files.c | 12 ++- builtin/pull.c | 2 + t/helper/test-read-cache.c | 64 ++--------- t/t1091-sparse-checkout-builtin.sh | 25 ++++- t/t1092-sparse-checkout-compatibility.sh | 132 ++++++++++++++++++++--- t/t3705-add-sparse-checkout.sh | 8 +- 8 files changed, 168 insertions(+), 82 deletions(-) base-commit: 597af311a2899bfd6640b9b107622c5795d5f998 Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1080%2Fderrickstolee%2Fsparse-index%2Ffetch-pull-ls-files-v4 Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1080/derrickstolee/sparse-index/fetch-pull-ls-files-v4 Pull-Request: https://github.com/gitgitgadget/git/pull/1080 Range-diff vs v3: 1: f72001638d1 = 1: 1043a1927d2 fetch/pull: use the sparse index 2: b81174ba54b ! 2: e1ec52881d9 ls-files: add --sparse option @@ Documentation/git-ls-files.txt: Both the <eolinfo> in the index ("i/<eolinfo>") +--sparse:: + If the index is sparse, show the sparse directories without expanding -+ to the contained files. ++ to the contained files. Sparse directories will be shown with a ++ trailing slash, such as "x/" for a sparse directory "x". + \--:: Do not interpret any more arguments as options. @@ builtin/ls-files.c: int cmd_ls_files(int argc, const char **argv, const char *cm ## t/t1092-sparse-checkout-compatibility.sh ## @@ t/t1092-sparse-checkout-compatibility.sh: test_expect_success 'sparse-index is expanded and converted back' ' - GIT_TRACE2_EVENT="$(pwd)/trace2.txt" GIT_TRACE2_EVENT_NESTING=10 \ + GIT_TRACE2_EVENT="$(pwd)/trace2.txt" \ git -C sparse-index reset -- folder1/a && test_region index convert_to_sparse trace2.txt && + test_region index ensure_full_index trace2.txt && @@ t/t1092-sparse-checkout-compatibility.sh: test_expect_success 'sparse index is n +test_expect_success 'ls-files' ' + init_repos && + ++ # Use a smaller sparse-checkout for reduced output ++ test_sparse_match git sparse-checkout set && ++ + # Behavior agrees by default. Sparse index is expanded. + test_all_match git ls-files && + + # With --sparse, the sparse index data changes behavior. -+ git -C sparse-index ls-files >dense && -+ git -C sparse-index ls-files --sparse >sparse && ++ git -C sparse-index ls-files --sparse >actual && + + cat >expect <<-\EOF && -+ @@ -13,13 +13,9 @@ -+ e -+ folder1- -+ folder1.x -+ -folder1/0/0/0 -+ -folder1/0/1 -+ -folder1/a -+ +folder1/ -+ folder10 -+ -folder2/0/0/0 -+ -folder2/0/1 -+ -folder2/a -+ +folder2/ -+ g -+ -x/a -+ +x/ -+ z ++ a ++ deep/ ++ e ++ folder1- ++ folder1.x ++ folder1/ ++ folder10 ++ folder2/ ++ g ++ x/ ++ z + EOF + -+ diff -u dense sparse | tail -n +3 >actual && + test_cmp expect actual && + + # With --sparse and no sparse index, nothing changes. @@ t/t1092-sparse-checkout-compatibility.sh: test_expect_success 'sparse index is n + test_sparse_match git sparse-checkout add folder1 && + test_sparse_match git ls-files --modified && + -+ git -C sparse-index ls-files >dense && -+ git -C sparse-index ls-files --sparse >sparse && ++ test_all_match git ls-files && ++ git -C sparse-index ls-files --sparse >actual && + + cat >expect <<-\EOF && -+ @@ -17,9 +17,7 @@ -+ folder1/0/1 -+ folder1/a -+ folder10 -+ -folder2/0/0/0 -+ -folder2/0/1 -+ -folder2/a -+ +folder2/ -+ g -+ -x/a -+ +x/ -+ z ++ a ++ deep/ ++ e ++ folder1- ++ folder1.x ++ folder1/0/0/0 ++ folder1/0/1 ++ folder1/a ++ folder10 ++ folder2/ ++ g ++ x/ ++ z + EOF + -+ diff -u dense sparse | tail -n +3 >actual && + test_cmp expect actual && + + # Double-check index expansion is avoided 3: 2a6a1c5a39c = 3: 0c53bd09ba4 t1092: replace 'read-cache --table' with 'ls-files --sparse' 4: f0143686754 = 4: 4952c9e724b t1091/t3705: remove 'test-tool read-cache --table' 5: 9227dc54165 = 5: 3efffad814c test-read-cache: remove --table, --expand options -- gitgitgadget
