As follow-up to [1], here's a series that eliminates redundant pipelines, mostly under t/. Spawning a process is notoriously slow on Windows, so this will improve test performance. 1/22 to 14/22 remove redundant uses of "cat" where the shell handles input redirection from a file or the command can take a file argument. 15/22 to 20/22 merge redundant uses of "grep" into the following "sed" or "awk" command. Both of these are capable of filtering themselves. 21/22 merges multiple "sed" calls into one and replaces a whole "cat | grep | awk" construct with a single "awk" call and uses "sort -u" instead of "sort | uniq". 22/22 finally is a bit of an outlier in that it replaces a subshell with a shell compound command. In the tests, I have completely arbitrarily separated the commits into thousands blocks to keep the commits and their reviews manageable. All tests that I was able to run still pass. I don't have p4 or cvs installed. [1] https://lore.kernel.org/git/20240216171046.927552-1-dev+git@xxxxxxxxx/ Changes from v1 (see also the range-diff below): - 09/22: leave a "cat" that concatenates two files as input to "git am" - 14/22: replace all instances of cat >expect.unsorted <<... && sort expect.unsorted >expect with "sort >expect <<..." - 19/22: avoid "sed -E" (use extended regex syntax) which may not be supported on each platform with equivalent basic regexes Beat Bolli (22): doc: avoid redundant use of cat contrib/subtree/t: avoid redundant use of cat t/lib-cvs.sh: avoid redundant use of cat t/annotate-tests.sh: avoid redundant use of cat t/perf: avoid redundant use of cat t/t0*: avoid redundant uses of cat t/t1*: avoid redundant uses of cat t/t3*: avoid redundant uses of cat t/t4*: avoid redundant uses of cat t/t5*: avoid redundant uses of cat t/t6*: avoid redundant uses of cat t/t7*: avoid redundant use of cat t/t8*: avoid redundant use of cat t/t9*: avoid redundant uses of cat t/t1*: merge a "grep | sed" pipeline t/t3*: merge a "grep | awk" pipeline t/t4*: merge a "grep | sed" pipeline t/t5*: merge a "grep | sed" pipeline t/t8*: merge "grep | sed" pipelines t/t9*: merge "grep | sed" pipelines contrib/coverage-diff: avoid redundant pipelines git-quiltimport: avoid an unnecessary subshell Documentation/howto/update-hook-example.txt | 4 +-- contrib/coverage-diff.sh | 9 ++---- contrib/subtree/t/t7900-subtree.sh | 2 +- git-quiltimport.sh | 2 +- t/annotate-tests.sh | 2 +- t/lib-cvs.sh | 4 +-- t/perf/repos/inflate-repo.sh | 2 +- t/t0002-gitfile.sh | 2 +- t/t0011-hashmap.sh | 2 +- t/t0028-working-tree-encoding.sh | 4 +-- t/t0204-gettext-reencode-sanity.sh | 2 +- t/t1007-hash-object.sh | 6 ++-- t/t1091-sparse-checkout-builtin.sh | 2 +- t/t1509/prepare-chroot.sh | 2 +- t/t3200-branch.sh | 2 +- t/t3321-notes-stripspace.sh | 8 ++--- t/t3920-crlf-messages.sh | 4 +-- t/t4002-diff-basic.sh | 2 +- t/t4020-diff-external.sh | 2 +- t/t4205-log-pretty-formats.sh | 2 +- t/t4301-merge-tree-write-tree.sh | 8 ++--- t/t5100-mailinfo.sh | 2 +- t/t5317-pack-objects-filter-objects.sh | 2 +- t/t5401-update-hooks.sh | 2 +- t/t5534-push-signed.sh | 2 +- t/t6112-rev-list-filters-objects.sh | 2 +- t/t6413-merge-crlf.sh | 4 +-- t/t7704-repack-cruft.sh | 2 +- t/t8010-cat-file-filters.sh | 2 +- t/t8013-blame-ignore-revs.sh | 28 ++++++++--------- t/t9118-git-svn-funky-branch-names.sh | 2 +- t/t9300-fast-import.sh | 14 +++------ t/t9350-fast-export.sh | 4 +-- t/t9400-git-cvsserver-server.sh | 35 ++++++++++----------- t/t9802-git-p4-filetype.sh | 2 +- t/t9807-git-p4-submit.sh | 2 +- t/t9824-git-p4-git-lfs.sh | 4 +-- 37 files changed, 85 insertions(+), 97 deletions(-) Range-diff from v1: 1: 270f19b35e36 = 1: 270f19b35e36 doc: avoid redundant use of cat 2: eed1fc58a7e4 = 2: eed1fc58a7e4 contrib/subtree/t: avoid redundant use of cat 3: c131707668ac = 3: c131707668ac t/lib-cvs.sh: avoid redundant use of cat 4: da69f859de63 = 4: da69f859de63 t/annotate-tests.sh: avoid redundant use of cat 5: 07b3bf33577f = 5: 07b3bf33577f t/perf: avoid redundant use of cat 6: 38907650b33e = 6: 38907650b33e t/t0*: avoid redundant uses of cat 7: b8b8b7ae47a6 = 7: b8b8b7ae47a6 t/t1*: avoid redundant uses of cat 8: 7c91c4092e7d = 8: 7c91c4092e7d t/t3*: avoid redundant uses of cat 9: 3c9df39774d8 ! 9: 1a80eaa97169 t/t4*: avoid redundant uses of cat @@ t/t4020-diff-external.sh: keep_only_cr () { test_expect_success 'diff --cached' ' - ## t/t4150-am.sh ## -@@ t/t4150-am.sh: test_expect_success 'am takes patches from a Pine mailbox' ' - rm -fr .git/rebase-apply && - git reset --hard && - git checkout first && -- cat pine patch1 | git am && -+ git am pine patch1 && - test_path_is_missing .git/rebase-apply && - git diff --exit-code main^..HEAD - ' - ## t/t4205-log-pretty-formats.sh ## @@ t/t4205-log-pretty-formats.sh: test_expect_success 'NUL termination with --reflog --pretty=oneline' ' for r in $revs 10: a3207b26a80e = 10: 3ac01bbcaafa t/t5*: avoid redundant uses of cat 11: bde50bfd2552 = 11: a4a6aaf4654b t/t6*: avoid redundant uses of cat 12: 84978c31631f = 12: 4b7b9df129a3 t/t7*: avoid redundant use of cat 13: a555a1809e84 = 13: ebf1e5eeadbd t/t8*: avoid redundant use of cat 14: 667c04d6c665 ! 14: 34cf66aeff80 t/t9*: avoid redundant uses of cat @@ t/t9300-fast-import.sh: test_expect_success 'L: nested tree copy does not corrup test_cmp expect actual && git fsck $(git rev-parse L2) ' -@@ t/t9300-fast-import.sh: test_expect_success 'Q: verify first notes tree' ' +@@ t/t9300-fast-import.sh: test_expect_success 'Q: verify first notes commit' ' + ' + + test_expect_success 'Q: verify first notes tree' ' +- cat >expect.unsorted <<-EOF && ++ sort >expect <<-EOF && + 100644 blob $commit1 100644 blob $commit2 100644 blob $commit3 EOF - cat expect.unsorted | sort >expect && -+ sort expect.unsorted >expect && git cat-file -p refs/notes/foobar~2^{tree} | sed "s/ [0-9a-f]* / /" >actual && test_cmp expect actual ' -@@ t/t9300-fast-import.sh: test_expect_success 'Q: verify second notes tree' ' +@@ t/t9300-fast-import.sh: test_expect_success 'Q: verify second notes commit' ' + ' + + test_expect_success 'Q: verify second notes tree' ' +- cat >expect.unsorted <<-EOF && ++ sort >expect <<-EOF && + 100644 blob $commit1 100644 blob $commit2 100644 blob $commit3 EOF - cat expect.unsorted | sort >expect && -+ sort expect.unsorted >expect && git cat-file -p refs/notes/foobar^^{tree} | sed "s/ [0-9a-f]* / /" >actual && test_cmp expect actual ' -@@ t/t9300-fast-import.sh: test_expect_success 'Q: verify third notes tree' ' - cat >expect.unsorted <<-EOF && +@@ t/t9300-fast-import.sh: test_expect_success 'Q: verify third notes commit' ' + ' + + test_expect_success 'Q: verify third notes tree' ' +- cat >expect.unsorted <<-EOF && ++ sort >expect <<-EOF && 100644 blob $commit1 EOF - cat expect.unsorted | sort >expect && -+ sort expect.unsorted >expect && git cat-file -p refs/notes/foobar2^{tree} | sed "s/ [0-9a-f]* / /" >actual && test_cmp expect actual ' -@@ t/t9300-fast-import.sh: test_expect_success 'Q: verify fourth notes tree' ' - cat >expect.unsorted <<-EOF && +@@ t/t9300-fast-import.sh: test_expect_success 'Q: verify fourth notes commit' ' + ' + + test_expect_success 'Q: verify fourth notes tree' ' +- cat >expect.unsorted <<-EOF && ++ sort >expect <<-EOF && 100644 blob $commit2 EOF - cat expect.unsorted | sort >expect && -+ sort expect.unsorted >expect && git cat-file -p refs/notes/foobar^{tree} | sed "s/ [0-9a-f]* / /" >actual && test_cmp expect actual ' 15: 6dd79d30c42a = 15: be300cb0a65a t/t1*: merge a "grep | sed" pipeline 16: db61133fa92f = 16: d6f5e21942ff t/t3*: merge a "grep | awk" pipeline 17: 5ce7ddeb2fbd = 17: 68a5f3f743d0 t/t4*: merge a "grep | sed" pipeline 18: 6ed68fe96493 = 18: 3a7e2a8fc388 t/t5*: merge a "grep | sed" pipeline 19: 698a5524b5c4 ! 19: e6ec4b3a0451 t/t8*: merge "grep | sed" pipelines @@ t/t8013-blame-ignore-revs.sh: test_expect_success setup ' git blame --line-porcelain file >blame_raw && - grep -E "^[0-9a-f]+ [0-9]+ 1" blame_raw | sed -e "s/ .*//" >actual && -+ sed -Ene "/^[0-9a-f]+ [0-9]+ 1/s/ .*//p" blame_raw >actual && ++ sed -ne "/^[0-9a-f][0-9a-f]* [0-9][0-9]* 1/s/ .*//p" blame_raw >actual && git rev-parse X >expect && test_cmp expect actual && - grep -E "^[0-9a-f]+ [0-9]+ 2" blame_raw | sed -e "s/ .*//" >actual && -+ sed -Ene "/^[0-9a-f]+ [0-9]+ 2/s/ .*//p" blame_raw >actual && ++ sed -ne "/^[0-9a-f][0-9a-f]* [0-9][0-9]* 2/s/ .*//p" blame_raw >actual && git rev-parse X >expect && test_cmp expect actual ' @@ t/t8013-blame-ignore-revs.sh: do git blame --line-porcelain --ignore-rev $I file >blame_raw && - grep -E "^[0-9a-f]+ [0-9]+ 1" blame_raw | sed -e "s/ .*//" >actual && -+ sed -Ene "/^[0-9a-f]+ [0-9]+ 1/s/ .*//p" blame_raw >actual && ++ sed -ne "/^[0-9a-f][0-9a-f]* [0-9][0-9]* 1/s/ .*//p" blame_raw >actual && git rev-parse A >expect && test_cmp expect actual && - grep -E "^[0-9a-f]+ [0-9]+ 2" blame_raw | sed -e "s/ .*//" >actual && -+ sed -Ene "/^[0-9a-f]+ [0-9]+ 2/s/ .*//p" blame_raw >actual && ++ sed -ne "/^[0-9a-f][0-9a-f]* [0-9][0-9]* 2/s/ .*//p" blame_raw >actual && git rev-parse B >expect && test_cmp expect actual ' @@ t/t8013-blame-ignore-revs.sh: test_expect_success ignore_rev_adding_unblamable_l git blame --line-porcelain file --ignore-rev Y >blame_raw && - grep -E "^[0-9a-f]+ [0-9]+ 3" blame_raw | sed -e "s/ .*//" >actual && -+ sed -Ene "/^[0-9a-f]+ [0-9]+ 3/s/ .*//p" blame_raw >actual && ++ sed -ne "/^[0-9a-f][0-9a-f]* [0-9][0-9]* 3/s/ .*//p" blame_raw >actual && test_cmp expect actual && - grep -E "^[0-9a-f]+ [0-9]+ 4" blame_raw | sed -e "s/ .*//" >actual && -+ sed -Ene "/^[0-9a-f]+ [0-9]+ 4/s/ .*//p" blame_raw >actual && ++ sed -ne "/^[0-9a-f][0-9a-f]* [0-9][0-9]* 4/s/ .*//p" blame_raw >actual && test_cmp expect actual ' @@ t/t8013-blame-ignore-revs.sh: test_expect_success ignore_revs_from_files ' git blame --line-porcelain file --ignore-revs-file ignore_x --ignore-revs-file ignore_y >blame_raw && - grep -E "^[0-9a-f]+ [0-9]+ 1" blame_raw | sed -e "s/ .*//" >actual && -+ sed -Ene "/^[0-9a-f]+ [0-9]+ 1/s/ .*//p" blame_raw >actual && ++ sed -ne "/^[0-9a-f][0-9a-f]* [0-9][0-9]* 1/s/ .*//p" blame_raw >actual && git rev-parse A >expect && test_cmp expect actual && - grep -E "^[0-9a-f]+ [0-9]+ 2" blame_raw | sed -e "s/ .*//" >actual && -+ sed -Ene "/^[0-9a-f]+ [0-9]+ 2/s/ .*//p" blame_raw >actual && ++ sed -ne "/^[0-9a-f][0-9a-f]* [0-9][0-9]* 2/s/ .*//p" blame_raw >actual && git rev-parse B >expect && test_cmp expect actual ' @@ t/t8013-blame-ignore-revs.sh: test_expect_success ignore_revs_from_configs_and_f git blame --line-porcelain file --ignore-revs-file ignore_y >blame_raw && - grep -E "^[0-9a-f]+ [0-9]+ 1" blame_raw | sed -e "s/ .*//" >actual && -+ sed -Ene "/^[0-9a-f]+ [0-9]+ 1/s/ .*//p" blame_raw >actual && ++ sed -ne "/^[0-9a-f][0-9a-f]* [0-9][0-9]* 1/s/ .*//p" blame_raw >actual && git rev-parse A >expect && test_cmp expect actual && - grep -E "^[0-9a-f]+ [0-9]+ 2" blame_raw | sed -e "s/ .*//" >actual && -+ sed -Ene "/^[0-9a-f]+ [0-9]+ 2/s/ .*//p" blame_raw >actual && ++ sed -ne "/^[0-9a-f][0-9a-f]* [0-9][0-9]* 2/s/ .*//p" blame_raw >actual && git rev-parse B >expect && test_cmp expect actual ' @@ t/t8013-blame-ignore-revs.sh: test_expect_success override_ignore_revs_file ' git rev-parse X >expect && - grep -E "^[0-9a-f]+ [0-9]+ 1" blame_raw | sed -e "s/ .*//" >actual && -+ sed -Ene "/^[0-9a-f]+ [0-9]+ 1/s/ .*//p" blame_raw >actual && ++ sed -ne "/^[0-9a-f][0-9a-f]* [0-9][0-9]* 1/s/ .*//p" blame_raw >actual && test_cmp expect actual && - grep -E "^[0-9a-f]+ [0-9]+ 2" blame_raw | sed -e "s/ .*//" >actual && -+ sed -Ene "/^[0-9a-f]+ [0-9]+ 2/s/ .*//p" blame_raw >actual && ++ sed -ne "/^[0-9a-f][0-9a-f]* [0-9][0-9]* 2/s/ .*//p" blame_raw >actual && test_cmp expect actual ' test_expect_success bad_files_and_revs ' @@ t/t8013-blame-ignore-revs.sh: test_expect_success ignore_merge ' git blame --line-porcelain file --ignore-rev M >blame_raw && - grep -E "^[0-9a-f]+ [0-9]+ 1" blame_raw | sed -e "s/ .*//" >actual && -+ sed -Ene "/^[0-9a-f]+ [0-9]+ 1/s/ .*//p" blame_raw >actual && ++ sed -ne "/^[0-9a-f][0-9a-f]* [0-9][0-9]* 1/s/ .*//p" blame_raw >actual && git rev-parse B >expect && test_cmp expect actual && - grep -E "^[0-9a-f]+ [0-9]+ 9" blame_raw | sed -e "s/ .*//" >actual && -+ sed -Ene "/^[0-9a-f]+ [0-9]+ 9/s/ .*//p" blame_raw >actual && ++ sed -ne "/^[0-9a-f][0-9a-f]* [0-9][0-9]* 9/s/ .*//p" blame_raw >actual && git rev-parse C >expect && test_cmp expect actual ' 20: 6b3345667040 = 20: 4772acb80163 t/t9*: merge "grep | sed" pipelines 21: 1784d5f3fdc1 = 21: a34653f91589 contrib/coverage-diff: avoid redundant pipelines 22: 4858de8ce44f = 22: 6c62dee0daa1 git-quiltimport: avoid an unnecessary subshell -- 2.44.0 From: Beat Bolli <dev+git@xxxxxxxxx> *** BLURB HERE *** Beat Bolli (22): doc: avoid redundant use of cat contrib/subtree/t: avoid redundant use of cat t/lib-cvs.sh: avoid redundant use of cat t/annotate-tests.sh: avoid redundant use of cat t/perf: avoid redundant use of cat t/t0*: avoid redundant uses of cat t/t1*: avoid redundant uses of cat t/t3*: avoid redundant uses of cat t/t4*: avoid redundant uses of cat t/t5*: avoid redundant uses of cat t/t6*: avoid redundant uses of cat t/t7*: avoid redundant use of cat t/t8*: avoid redundant use of cat t/t9*: avoid redundant uses of cat t/t1*: merge a "grep | sed" pipeline t/t3*: merge a "grep | awk" pipeline t/t4*: merge a "grep | sed" pipeline t/t5*: merge a "grep | sed" pipeline t/t8*: merge "grep | sed" pipelines t/t9*: merge "grep | sed" pipelines contrib/coverage-diff: avoid redundant pipelines git-quiltimport: avoid an unnecessary subshell Documentation/howto/update-hook-example.txt | 4 +-- contrib/coverage-diff.sh | 9 ++---- contrib/subtree/t/t7900-subtree.sh | 2 +- git-quiltimport.sh | 2 +- t/annotate-tests.sh | 2 +- t/lib-cvs.sh | 4 +-- t/perf/repos/inflate-repo.sh | 2 +- t/t0002-gitfile.sh | 2 +- t/t0011-hashmap.sh | 2 +- t/t0028-working-tree-encoding.sh | 4 +-- t/t0204-gettext-reencode-sanity.sh | 2 +- t/t1007-hash-object.sh | 6 ++-- t/t1091-sparse-checkout-builtin.sh | 2 +- t/t1509/prepare-chroot.sh | 2 +- t/t3200-branch.sh | 2 +- t/t3321-notes-stripspace.sh | 8 ++--- t/t3920-crlf-messages.sh | 4 +-- t/t4002-diff-basic.sh | 2 +- t/t4020-diff-external.sh | 2 +- t/t4205-log-pretty-formats.sh | 2 +- t/t4301-merge-tree-write-tree.sh | 8 ++--- t/t5100-mailinfo.sh | 2 +- t/t5317-pack-objects-filter-objects.sh | 2 +- t/t5401-update-hooks.sh | 2 +- t/t5534-push-signed.sh | 2 +- t/t6112-rev-list-filters-objects.sh | 2 +- t/t6413-merge-crlf.sh | 4 +-- t/t7704-repack-cruft.sh | 2 +- t/t8010-cat-file-filters.sh | 2 +- t/t8013-blame-ignore-revs.sh | 28 ++++++++--------- t/t9118-git-svn-funky-branch-names.sh | 2 +- t/t9300-fast-import.sh | 14 +++------ t/t9350-fast-export.sh | 4 +-- t/t9400-git-cvsserver-server.sh | 35 ++++++++++----------- t/t9802-git-p4-filetype.sh | 2 +- t/t9807-git-p4-submit.sh | 2 +- t/t9824-git-p4-git-lfs.sh | 4 +-- 37 files changed, 85 insertions(+), 97 deletions(-) -- 2.44.0
