This series completes the initial implementation of the Scalar command as a
core component of Git. For readers new to the topic of Scalar, the
roadmap/design doc [1] provides some background information including how
the project started & evolved, as well as its current intent & high-level
design.
Changes since V1
================
* Updated 'scalar.txt' description from "Associated with [git]" -> "Part of
[git]".
* Fixed Dscho's "Signed-off-by" and author emails on patches 3 & 4.
* Added 'scalar' to 'git help -a' command list.
* Removed translation tag for literal 'scalar help' usage string.
* Updated 'scalar clone' tests to clone to unique targets and clean up only
when test passes.
* Changed 'p9210' perf tests to invoke 'git' and 'scalar' with '-C', rather
than creating a subshell to run commands.
* Fixed use of 'test_env_bool' with 'GIT_PERF_USE_SCALAR', locally tested
to make sure it was working as-intended.
Goals
=====
The goal of this series is, as noted in the roadmap linked earlier, to
integrate Scalar as a core user-facing component of Git. There's no
individual existing precedent that the integration is 100% based on; it's
more of an amalgam of the approaches used with builtins installed directly
to the bindir (e.g. 'git-upload-pack'), contributed tools (e.g. 'gitk'), and
the 'git' executable itself. The resulting behavior is:
* Scalar is built & installed unconditionally (like a builtin).
* Scalar is included in 'bin-wrappers/' and is installed into the 'bin/'
directory (like 'git-upload-pack').
* Scalar's documentation is installed as 'man1' docs (like 'gitk').
* Scalar is not invoked via the 'git' executable (it's called directly,
like 'git' itself).
The move out of 'contrib/' is also intended to mean that the relationship
between Git and Scalar should no longer be a strict one-way dependency. In
this series, this architectural change is what permits making a change to
'help.c' specifically for the sake of Scalar. Future contributions can and
should (where appropriate!) share code between Scalar and non-Scalar parts
of Git.
Organization
============
The series is broken up as follows:
1. Fix a CI-breaking documentation error in 'scalar.txt' (patch 1).
2. Move Scalar code out of 'contrib/' and into core Git, including changes
in build definition(s) to build & install Scalar and its resources
(patch 2).
3. Make 'git help scalar' open the 'scalar' command documentation,
introduce 'scalar help' as an alias. This is done after moving out of
'contrib/' due to it involving Scalar-related changes to the non-Scalar
file 'help.c' (patches 3-4).
4. Expand test coverage, including perf testing (patches 5-6)
5. Add a perf test configuration allowing users to register standard perf
test repos with Scalar (patch 7).
6. Remove the internally-focused Scalar roadmap from
'Documentation/technical/scalar.txt', include it as one of the HTML
technical docs (patch 8).
Prior series
============
* Add 'scalar' command in 'contrib/':
https://lore.kernel.org/git/pull.1005.v10.git.1638538470.gitgitgadget@xxxxxxxxx/
* Introduce 'scalar diagnose':
https://lore.kernel.org/git/pull.1128.v6.git.1653145696.gitgitgadget@xxxxxxxxx/
* Add '-c/-C' compatibility:
https://lore.kernel.org/git/pull.1130.v2.git.1643380317358.gitgitgadget@xxxxxxxxx/
* [DROPPED] Integrate Scalar into CI builds:
https://lore.kernel.org/git/pull.1129.git.1654160735.gitgitgadget@xxxxxxxxx/
* Document Scalar's role in Git & plan remaining work:
https://lore.kernel.org/git/pull.1275.v2.git.1657584367.gitgitgadget@xxxxxxxxx/
* Generalize 'scalar diagnose' into 'git diagnose' builtin & 'git bugreport
--diagnose':
https://lore.kernel.org/git/pull.1310.v4.git.1660335019.gitgitgadget@xxxxxxxxx/
* Add FSMonitor support to Scalar, refactor enlistment search:
https://lore.kernel.org/git/pull.1324.v3.git.1660858853.gitgitgadget@xxxxxxxxx/
Thanks!
* Victoria
[1]
https://lore.kernel.org/git/65eda9755d2a29e222bf7e6cc719921b24377cc6.1657584367.git.gitgitgadget@xxxxxxxxx/
Johannes Schindelin (2):
git help: special-case `scalar`
scalar: implement the `help` subcommand
Victoria Dye (7):
scalar: fix command documentation section header
scalar: include in standard Git build & installation
scalar: add to 'git help -a' command list
scalar-clone: add test coverage
t/perf: add Scalar performance tests
t/perf: add 'GIT_PERF_USE_SCALAR' run option
Documentation/technical: include Scalar technical doc
.gitignore | 1 +
Documentation/Makefile | 2 +
Documentation/cmd-list.perl | 2 +-
{contrib/scalar => Documentation}/scalar.txt | 4 +-
Documentation/technical/scalar.txt | 61 -------
Makefile | 31 ++--
builtin/help.c | 2 +
command-list.txt | 1 +
contrib/buildsystems/CMakeLists.txt | 9 +-
contrib/scalar/.gitignore | 2 -
contrib/scalar/Makefile | 35 ----
contrib/scalar/t/Makefile | 81 ----------
contrib/scalar/scalar.c => scalar.c | 20 +++
t/perf/README | 4 +
t/perf/p9210-scalar.sh | 39 +++++
t/perf/perf-lib.sh | 13 +-
t/perf/run | 3 +
.../t/t9099-scalar.sh => t/t9210-scalar.sh | 10 +-
t/t9211-scalar-clone.sh | 151 ++++++++++++++++++
19 files changed, 264 insertions(+), 207 deletions(-)
rename {contrib/scalar => Documentation}/scalar.txt (99%)
delete mode 100644 contrib/scalar/.gitignore
delete mode 100644 contrib/scalar/Makefile
delete mode 100644 contrib/scalar/t/Makefile
rename contrib/scalar/scalar.c => scalar.c (98%)
create mode 100755 t/perf/p9210-scalar.sh
rename contrib/scalar/t/t9099-scalar.sh => t/t9210-scalar.sh (96%)
create mode 100755 t/t9211-scalar-clone.sh
base-commit: d42b38dfb5edf1a7fddd9542d722f91038407819
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1341%2Fvdye%2Ffeature%2Fscalar-toplevel-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1341/vdye/feature/scalar-toplevel-v2
Pull-Request: https://github.com/gitgitgadget/git/pull/1341
Range-diff vs v1:
1: fcf73e77574 ! 1: 614cbdc629d scalar: fix command documentation section header
@@ contrib/scalar/scalar.txt: SEE ALSO
-Scalar
+GIT
---
- Associated with the linkgit:git[1] suite
+-Associated with the linkgit:git[1] suite
++Part of the linkgit:git[1] suite
2: 4d69e5eaccb ! 2: bc2092a7a7a scalar: include in standard Git build & installation
@@ Makefile: XGETTEXT_FLAGS_SH = $(XGETTEXT_FLAGS) --language=Shell \
--keyword=__ --keyword=N__ --keyword="__n:1,2"
MSGMERGE_FLAGS = --add-location --backup=off --update
-LOCALIZED_C = $(sort $(FOUND_C_SOURCES) $(FOUND_H_SOURCES) $(SCALAR_SOURCES) \
-+LOCALIZED_C = $(sort $(FOUND_C_SOURCES) $(FOUND_H_SOURCES) \
- $(GENERATED_H))
+- $(GENERATED_H))
++LOCALIZED_C = $(sort $(FOUND_C_SOURCES) $(FOUND_H_SOURCES) $(GENERATED_H))
LOCALIZED_SH = $(sort $(SCRIPT_SH) git-sh-setup.sh)
LOCALIZED_PERL = $(sort $(SCRIPT_PERL))
+
@@ Makefile: bin-wrappers/%: wrap-for-bin.sh
$(call mkdir_p_parent_template)
$(QUIET_GEN)sed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \
3: 969160650aa = 3: eb8663e0300 git help: special-case `scalar`
4: 46d0fddfe8f ! 4: 55aba052c34 scalar: implement the `help` subcommand
@@
## Metadata ##
-Author: Johannes Schindelin <johasc@xxxxxxxxxxxxx>
+Author: Johannes Schindelin <Johannes.Schindelin@xxxxxx>
## Commit message ##
scalar: implement the `help` subcommand
It is merely handing off to `git help scalar`.
- Signed-off-by: Johannes Schindelin <johasc@xxxxxxxxxxxxx>
+ Signed-off-by: Johannes Schindelin <johannes.schindelin@xxxxxx>
Signed-off-by: Victoria Dye <vdye@xxxxxxxxxx>
## scalar.c ##
@@ scalar.c: static int cmd_delete(int argc, const char **argv)
+ OPT_END(),
+ };
+ const char * const usage[] = {
-+ N_("scalar help"),
++ "scalar help",
+ NULL
+ };
+
-: ----------- > 5: 070f195f027 scalar: add to 'git help -a' command list
5: a3b7cb0a3bd ! 6: 15acf20274b scalar-clone: add test coverage
@@ Commit message
scalar-clone: add test coverage
Create a new test file ('t9211-scalar-clone.sh') to exercise the options and
- behavior of the 'scalar clone' command.
+ behavior of the 'scalar clone' command. Each test clones to a unique target
+ location and cleans up the cloned repo only when the test passes. This
+ ensures that failed tests' artifacts are captured in CI artifacts for
+ further debugging.
+ Helped-by: Johannes Schindelin <johannes.schindelin@xxxxxx>
Signed-off-by: Victoria Dye <vdye@xxxxxxxxxx>
## t/t9211-scalar-clone.sh (new) ##
@@ t/t9211-scalar-clone.sh (new)
+}
+
+test_expect_success 'creates content in enlistment root' '
-+ test_when_finished cleanup_clone cloned &&
++ enlistment=cloned &&
+
-+ scalar clone "file://$(pwd)/to-clone" cloned &&
-+ ls -A cloned >enlistment-root &&
++ scalar clone "file://$(pwd)/to-clone" $enlistment &&
++ ls -A $enlistment >enlistment-root &&
+ test_line_count = 1 enlistment-root &&
-+ test_path_is_dir cloned/src &&
-+ test_path_is_dir cloned/src/.git
++ test_path_is_dir $enlistment/src &&
++ test_path_is_dir $enlistment/src/.git &&
++
++ cleanup_clone $enlistment
+'
+
+test_expect_success 'with spaces' '
-+ test_when_finished cleanup_clone "cloned with space" &&
++ enlistment="cloned with space" &&
++
++ scalar clone "file://$(pwd)/to-clone" "$enlistment" &&
++ test_path_is_dir "$enlistment" &&
++ test_path_is_dir "$enlistment/src" &&
++ test_path_is_dir "$enlistment/src/.git" &&
+
-+ scalar clone "file://$(pwd)/to-clone" "cloned with space" &&
-+ test_path_is_dir "cloned with space" &&
-+ test_path_is_dir "cloned with space/src" &&
-+ test_path_is_dir "cloned with space/src/.git"
++ cleanup_clone "$enlistment"
+'
+
+test_expect_success 'partial clone if supported by server' '
-+ test_when_finished cleanup_clone cloned &&
++ enlistment=partial-clone &&
+
-+ scalar clone "file://$(pwd)/to-clone" cloned &&
++ scalar clone "file://$(pwd)/to-clone" $enlistment &&
+
+ (
-+ cd cloned/src &&
++ cd $enlistment/src &&
+
+ # Two promisor packs: one for refs, the other for blobs
+ ls .git/objects/pack/pack-*.promisor >promisorlist &&
+ test_line_count = 2 promisorlist
-+ )
++ ) &&
++
++ cleanup_clone $enlistment
+'
+
+test_expect_success 'fall back on full clone if partial unsupported' '
-+ test_when_finished cleanup_clone cloned &&
++ enlistment=no-partial-support &&
+
+ test_config -C to-clone uploadpack.allowfilter false &&
+ test_config -C to-clone uploadpack.allowanysha1inwant false &&
+
-+ scalar clone "file://$(pwd)/to-clone" cloned 2>err &&
++ scalar clone "file://$(pwd)/to-clone" $enlistment 2>err &&
+ grep "filtering not recognized by server, ignoring" err &&
+
+ (
-+ cd cloned/src &&
++ cd $enlistment/src &&
+
+ # Still get a refs promisor file, but none for blobs
+ ls .git/objects/pack/pack-*.promisor >promisorlist &&
+ test_line_count = 1 promisorlist
-+ )
++ ) &&
++
++ cleanup_clone $enlistment
+'
+
+test_expect_success 'initializes sparse-checkout by default' '
-+ test_when_finished cleanup_clone cloned &&
++ enlistment=sparse &&
+
-+ scalar clone "file://$(pwd)/to-clone" cloned &&
++ scalar clone "file://$(pwd)/to-clone" $enlistment &&
+ (
-+ cd cloned/src &&
++ cd $enlistment/src &&
+ test_cmp_config true core.sparseCheckout &&
+ test_cmp_config true core.sparseCheckoutCone
-+ )
++ ) &&
++
++ cleanup_clone $enlistment
+'
+
+test_expect_success '--full-clone does not create sparse-checkout' '
-+ test_when_finished cleanup_clone cloned &&
++ enlistment=full-clone &&
+
-+ scalar clone --full-clone "file://$(pwd)/to-clone" cloned &&
++ scalar clone --full-clone "file://$(pwd)/to-clone" $enlistment &&
+ (
-+ cd cloned/src &&
++ cd $enlistment/src &&
+ test_cmp_config "" --default "" core.sparseCheckout &&
+ test_cmp_config "" --default "" core.sparseCheckoutCone
-+ )
++ ) &&
++
++ cleanup_clone $enlistment
+'
+
+test_expect_success '--single-branch clones HEAD only' '
-+ test_when_finished cleanup_clone cloned &&
++ enlistment=single-branch &&
+
-+ scalar clone --single-branch "file://$(pwd)/to-clone" cloned &&
++ scalar clone --single-branch "file://$(pwd)/to-clone" $enlistment &&
+ (
-+ cd cloned/src &&
++ cd $enlistment/src &&
+ git for-each-ref refs/remotes/origin >out &&
+ test_line_count = 1 out &&
+ grep "refs/remotes/origin/base" out
-+ )
++ ) &&
++
++ cleanup_clone $enlistment
+'
+
+test_expect_success '--no-single-branch clones all branches' '
-+ test_when_finished cleanup_clone cloned &&
++ enlistment=no-single-branch &&
+
-+ scalar clone --no-single-branch "file://$(pwd)/to-clone" cloned &&
++ scalar clone --no-single-branch "file://$(pwd)/to-clone" $enlistment &&
+ (
-+ cd cloned/src &&
++ cd $enlistment/src &&
+ git for-each-ref refs/remotes/origin >out &&
+ test_line_count = 2 out &&
+ grep "refs/remotes/origin/base" out &&
+ grep "refs/remotes/origin/parallel" out
-+ )
++ ) &&
++
++ cleanup_clone $enlistment
+'
+
+test_done
6: 42ab39f2121 ! 7: d5b68c65163 t/perf: add Scalar performance tests
@@ Commit message
9210.6: test_commit --append --no-tag A (scalar) 0.08(0.02+0.03)
9210.7: test_commit --append --no-tag A (non-scalar) 0.13(0.03+0.11)
+ Helped-by: Johannes Schindelin <johannes.schindelin@xxxxxx>
Signed-off-by: Victoria Dye <vdye@xxxxxxxxxx>
## t/perf/p9210-scalar.sh (new) ##
@@ t/perf/p9210-scalar.sh (new)
+'
+
+test_compare_perf () {
-+ command="$@"
-+ test_perf "$command (scalar)" "
-+ (
-+ cd scalar-clone/src &&
-+ $command
-+ )
++ command=$1
++ shift
++ args=$*
++ test_perf "$command $args (scalar)" "
++ $command -C scalar-clone/src $args
+ "
+
-+ test_perf "$command (non-scalar)" "
-+ (
-+ cd git-clone &&
-+ $command
-+ )
++ test_perf "$command $args (non-scalar)" "
++ $command -C git-clone $args
+ "
+}
+
7: 96e08a95265 ! 8: 2e6dd03beba t/perf: add 'GIT_PERF_USE_SCALAR' run option
@@ t/perf/perf-lib.sh: test_perf_create_repo_from () {
# a copy it's fine to remove the lock.
rm .git/index.lock
+ fi &&
-+ if test_bool_env "$GIT_PERF_USE_SCALAR" false
++ if test_bool_env GIT_PERF_USE_SCALAR false
+ then
+ "$MODERN_SCALAR" register
fi
@@ t/perf/perf-lib.sh: test_perf_fresh_repo () {
cd "$repo" &&
- test_perf_do_repo_symlink_config_
+ test_perf_do_repo_symlink_config_ &&
-+ if test_bool_env "$GIT_PERF_USE_SCALAR" false
++ if test_bool_env GIT_PERF_USE_SCALAR false
+ then
+ "$MODERN_SCALAR" register
+ fi
8: 8bbe913dccf = 9: 6bedd80c398 Documentation/technical: include Scalar technical doc
--
gitgitgadget
