This is the final part required for the stage 4 implementation of
SHA-256.
Changes from v2:
* Move bundle signature parsing into a function.
* Improve parsing of bundle capabilities.
* Improve t5510 to select only the interesting lines with sed.
* Use cat and a heredoc instead of multiple echos.
* Use test_i18ngrep where appropriate.
* Reject extensions.objectFormat for repo v0.
* Update documentation accordingly.
Changes from v1:
* Add a patch to add a bundle v3 format with capabilities that include
the hash algorithm.
* Remove bundle detection of hash algorithm using hash algorithm length.
* Fix typo in the extensions.objectFormat documentation.
* Document that extensions.objectFormat is ignored for repo v0.
* Use test instead of brackets.
* Use a command group instead of a subshell for efficiency.
* Drop the bizarre test_might_fail, since it isn't needed (and is
bizarre and wrong).
* Write "SHA-256" instead of "NewHash" in commit messages.
* Split SHA-256 CI tests to an independent run and avoid running a third
test run to a single CI job.
Johannes Schindelin (1):
t3404: prepare 'short SHA-1 collision' tests for SHA-256
brian m. carlson (38):
t: make test-bloom initialize repository
t1001: use $ZERO_OID
t3305: make hash agnostic
t6100: make hash size independent
t6101: make hash size independent
t6301: make hash size independent
t6500: specify test values for SHA-256
t6501: avoid hard-coded objects
t7003: compute appropriate length constant
t7063: make hash size independent
t7201: abstract away SHA-1-specific constants
t7102: abstract away SHA-1-specific constants
t7400: make hash size independent
t7405: make hash size independent
t7506: avoid checking for SHA-1-specific constants
t7508: use $ZERO_OID instead of hard-coded constant
t8002: make hash size independent
t8003: make hash size independent
t8011: make hash size independent
t9300: abstract away SHA-1-specific constants
t9300: use $ZERO_OID instead of hard-coded object ID
t9301: make hash size independent
t9350: make hash size independent
t9500: ensure that algorithm info is preserved in config
t9700: make hash size independent
t5308: make test work with SHA-256
t0410: mark test with SHA1 prerequisite
http-fetch: set up git directory before parsing pack hashes
builtin/verify-pack: implement an --object-format option
bundle: add new version for use with SHA-256
setup: add support for reading extensions.objectformat
Enable SHA-256 support by default
t: add test_oid option to select hash algorithm
t: allow testing different hash algorithms via environment
t: make SHA1 prerequisite depend on default hash
ci: run tests with SHA-256
docs: add documentation for extensions.objectFormat
t: remove test_oid_init in tests
Documentation/config.txt | 2 +
Documentation/config/extensions.txt | 8 ++
Documentation/git-bundle.txt | 9 +-
Documentation/technical/bundle-format.txt | 30 ++++-
builtin/bundle.c | 5 +-
builtin/init-db.c | 5 -
builtin/verify-pack.c | 25 ++--
bundle.c | 85 ++++++++----
bundle.h | 4 +-
ci/run-build-and-tests.sh | 5 +
config.mak.dev | 2 -
http-fetch.c | 4 +-
repository.c | 4 -
setup.c | 16 ++-
t/helper/test-bloom.c | 2 +
t/lib-pack.sh | 11 +-
t/lib-submodule-update.sh | 1 -
t/t0000-basic.sh | 15 ++-
t/t0001-init.sh | 29 +++++
t/t0410-partial-clone.sh | 4 +-
t/t1006-cat-file.sh | 2 -
t/t1050-large.sh | 1 -
t/t1091-sparse-checkout-builtin.sh | 4 +-
t/t1410-reflog.sh | 1 -
t/t1450-fsck.sh | 1 -
t/t1500-rev-parse.sh | 1 -
t/t3305-notes-fanout.sh | 2 +-
t/t3308-notes-merge.sh | 1 -
t/t3404-rebase-interactive.sh | 49 +++++--
t/t3600-rm.sh | 1 -
t/t3800-mktag.sh | 1 -
t/t4002-diff-basic.sh | 2 -
t/t4027-diff-submodule.sh | 1 -
t/t4134-apply-submodule.sh | 1 -
t/t4200-rerere.sh | 1 -
t/t4211-line-log.sh | 1 -
t/t5300-pack-object.sh | 3 +-
t/t5302-pack-index.sh | 1 -
t/t5308-pack-detect-duplicates.sh | 20 +--
t/t5313-pack-bounds-checks.sh | 1 -
t/t5318-commit-graph.sh | 3 +-
t/t5319-multi-pack-index.sh | 1 -
t/t5324-split-commit-graph.sh | 1 -
t/t5504-fetch-receive-strict.sh | 1 -
t/t5510-fetch.sh | 11 +-
t/t5530-upload-pack-error.sh | 1 -
t/t5562-http-backend-content-length.sh | 1 -
t/t5607-clone-bundle.sh | 29 +++++
t/t5702-protocol-v2.sh | 3 +-
t/t5703-upload-pack-ref-in-want.sh | 1 -
t/t6006-rev-list-format.sh | 1 -
t/t6100-rev-list-in-order.sh | 4 +-
t/t6101-rev-parse-parents.sh | 2 +-
t/t6301-for-each-ref-errors.sh | 2 +-
t/t6500-gc.sh | 27 +++-
t/t6501-freshen-objects.sh | 14 +-
t/t7003-filter-branch.sh | 3 +-
t/t7063-status-untracked-cache.sh | 151 ++++++++++++----------
t/t7102-reset.sh | 93 +++++++------
t/t7201-co.sh | 5 +-
t/t7400-submodule-basic.sh | 26 ++--
t/t7405-submodule-merge.sh | 4 +-
t/t7506-status-submodule.sh | 12 +-
t/t7508-status.sh | 2 +-
t/t8002-blame.sh | 15 ++-
t/t8003-blame-corner-cases.sh | 3 +-
t/t8011-blame-split-file.sh | 2 +-
t/t9300-fast-import.sh | 117 +++++++++--------
t/t9301-fast-import-notes.sh | 13 +-
t/t9350-fast-export.sh | 15 +--
t/t9500-gitweb-standalone-no-errors.sh | 22 +++-
t/t9700/test.pl | 6 +-
t/test-lib-functions.sh | 16 ++-
t/test-lib.sh | 6 +-
74 files changed, 620 insertions(+), 354 deletions(-)
create mode 100644 Documentation/config/extensions.txt
Range-diff against v2:
1: 11bb3918df = 1: 792a7e6b99 t: make test-bloom initialize repository
2: bff32965d2 = 2: 652259edaa t1001: use $ZERO_OID
3: 03f49899e0 = 3: b719726393 t3305: make hash agnostic
4: d6c75da18f = 4: 137c86a4e0 t3404: prepare 'short SHA-1 collision' tests for SHA-256
5: 31cbce9d44 = 5: 307b90fa2a t6100: make hash size independent
6: bc21adaf33 = 6: f5f4028291 t6101: make hash size independent
7: 31a1c614cf = 7: 654b95494c t6301: make hash size independent
8: 4c19b39507 = 8: 787bde1ffc t6500: specify test values for SHA-256
9: cd43450d20 = 9: b63ed599fe t6501: avoid hard-coded objects
10: 2f7ef3bd9d = 10: 40847a1490 t7003: compute appropriate length constant
11: 96b29887b0 = 11: 3dbd65993c t7063: make hash size independent
12: d67c1e49d0 = 12: bba7969017 t7201: abstract away SHA-1-specific constants
13: 48b254250b = 13: bf5d06f1fc t7102: abstract away SHA-1-specific constants
14: cfc96d8181 = 14: a272eafa86 t7400: make hash size independent
15: 187f3bf5c1 = 15: 4dc14d1ded t7405: make hash size independent
16: 0d91991588 = 16: f291df2f20 t7506: avoid checking for SHA-1-specific constants
17: e22ec03788 = 17: ded35216ce t7508: use $ZERO_OID instead of hard-coded constant
18: bfd9a593a7 = 18: a197d7f09a t8002: make hash size independent
19: d4ca86e473 = 19: 7c49d9a8e8 t8003: make hash size independent
20: 28d1b7bf0b = 20: 4ab13f691a t8011: make hash size independent
21: c982602ffa = 21: 81edf64b72 t9300: abstract away SHA-1-specific constants
22: 503be79740 = 22: d8253d9aca t9300: use $ZERO_OID instead of hard-coded object ID
23: bdede03ff0 = 23: 7b40da5753 t9301: make hash size independent
24: a50d67d7bd = 24: 1890f22a5b t9350: make hash size independent
25: acea64aa18 = 25: 708f599a71 t9500: ensure that algorithm info is preserved in config
26: 0b5bb05dea = 26: 33c84a2bc6 t9700: make hash size independent
27: 41a1e3a78f = 27: 3d7476f28b t5308: make test work with SHA-256
28: 70f77a2508 < -: ---------- t0410: mark test with SHA1 prerequisite
-: ---------- > 28: 0d80e419e5 t0410: mark test with SHA1 prerequisite
29: 1c619ed160 = 29: bd812fb0e4 http-fetch: set up git directory before parsing pack hashes
30: cd643a0b1d = 30: 667537e1aa builtin/verify-pack: implement an --object-format option
31: 611ad8ffd0 ! 31: acdf1d9009 bundle: add new version for use with SHA-256
@@ Commit message
for SHA-256.
In t5510, always use format v3 so we can be sure we produce consistent
- results across hash algorithms.
+ results across hash algorithms. Note here that head -n N lists the top
+ N lines, whereas we want to check only the given line, so while we're at
+ it, switch to use sed to make sure we're checking only the line we care
+ about for a match.
Signed-off-by: brian m. carlson <sandals@xxxxxxxxxxxxxxxxxxxx>
@@ bundle.c
#include "argv-array.h"
-static const char bundle_signature[] = "# v2 git bundle\n";
++
+static const char v2_bundle_signature[] = "# v2 git bundle\n";
+static const char v3_bundle_signature[] = "# v3 git bundle\n";
++static struct {
++ int version;
++ const char *signature;
++} bundle_sigs[] = {
++ { 2, v2_bundle_signature },
++ { 3, v3_bundle_signature },
++};
static void add_to_ref_list(const struct object_id *oid, const char *name,
struct ref_list *list)
@@ bundle.c: static void add_to_ref_list(const struct object_id *oid, const char *n
{
- size_t len = strcspn(buf->buf, " \n");
- int algo;
--
-- algo = hash_algo_by_length(len / 2);
-- if (algo == GIT_HASH_UNKNOWN)
-- return NULL;
-- return &hash_algos[algo];
+ const char *arg;
+ if (skip_prefix(capability, "object-format=", &arg)) {
+ int algo = hash_algo_by_name(arg);
@@ bundle.c: static void add_to_ref_list(const struct object_id *oid, const char *n
+ return 0;
+ }
+ return error(_("unknown capability '%s'"), capability);
++}
+
+- algo = hash_algo_by_length(len / 2);
+- if (algo == GIT_HASH_UNKNOWN)
+- return NULL;
+- return &hash_algos[algo];
++static int parse_bundle_signature(struct bundle_header *header, const char *line)
++{
++ int i;
++
++ for (i = 0; i < ARRAY_SIZE(bundle_sigs); i++) {
++ if (!strcmp(line, bundle_sigs[i].signature)) {
++ header->version = bundle_sigs[i].version;
++ return 0;
++ }
++ }
++ return -1;
}
-+
static int parse_bundle_header(int fd, struct bundle_header *header,
- const char *report_path)
- {
@@ bundle.c: static int parse_bundle_header(int fd, struct bundle_header *header,
/* The bundle header begins with the signature */
if (strbuf_getwholeline_fd(&buf, fd, '\n') ||
- strcmp(buf.buf, bundle_signature)) {
-+ (strcmp(buf.buf, v2_bundle_signature) &&
-+ strcmp(buf.buf, v3_bundle_signature))) {
++ parse_bundle_signature(header, buf.buf)) {
if (report_path)
- error(_("'%s' does not look like a v2 bundle file"),
+ error(_("'%s' does not look like a v2 or v3 bundle file"),
@@ bundle.c: static int parse_bundle_header(int fd, struct bundle_header *header,
goto abort;
}
-+ header->version = buf.buf[3] - '0';
+ header->hash_algo = the_hash_algo;
+
/* The bundle header ends with an empty line */
@@ bundle.c: static int parse_bundle_header(int fd, struct bundle_header *header,
const char *p;
+ if (header->version == 3 && *buf.buf == '@') {
-+ buf.buf[buf.len - 1] = '\0';
++ buf.buf[--buf.len] = '\0';
+ if (parse_capability(header, buf.buf + 1)) {
+ status = -1;
+ break;
@@ t/t5510-fetch.sh: test_expect_success 'create bundle 1' '
test_expect_success 'header of bundle looks right' '
head -n 4 "$D"/bundle1 &&
- head -n 1 "$D"/bundle1 | grep "^#" &&
+- head -n 1 "$D"/bundle1 | grep "^#" &&
- head -n 2 "$D"/bundle1 | grep "^-$OID_REGEX " &&
- head -n 3 "$D"/bundle1 | grep "^$OID_REGEX " &&
- head -n 4 "$D"/bundle1 | grep "^$"
-+ head -n 2 "$D"/bundle1 | grep "^@object-format=" &&
-+ head -n 3 "$D"/bundle1 | grep "^-$OID_REGEX " &&
-+ head -n 4 "$D"/bundle1 | grep "^$OID_REGEX " &&
-+ head -n 5 "$D"/bundle1 | grep "^$"
++ sed -n -e 1p "$D"/bundle1 | grep "^#" &&
++ sed -n -e 2p "$D"/bundle1 | grep "^@object-format=" &&
++ sed -n -e 3p "$D"/bundle1 | grep "^-$OID_REGEX " &&
++ sed -n -e 4p "$D"/bundle1 | grep "^$OID_REGEX " &&
++ sed -n -e 5p "$D"/bundle1 | grep "^$"
'
test_expect_success 'create bundle 2' '
@@ t/t5607-clone-bundle.sh: test_expect_success 'fetch SHA-1 from bundle' '
+ git branch side HEAD &&
+ git bundle create --version=3 bundle HEAD^..side &&
+ head -n2 bundle >actual &&
-+ echo "# v3 git bundle" >expected &&
-+ echo "@object-format=$(test_oid algo)" >>expected &&
-+ test_cmp actual expected &&
++ cat >expect <<-EOF &&
++ # v3 git bundle
++ @object-format=$(test_oid algo)
++ EOF
++ test_cmp actual expect &&
+ git bundle verify bundle
+'
+
+test_expect_success 'git bundle v3 rejects unknown extensions' '
+ head -n2 bundle >new &&
+ echo "@unknown=silly" >>new &&
-+ sed "1,2d" >>new &&
++ sed "1,2d" bundle >>new &&
+ test_must_fail git bundle verify new 2>output &&
-+ grep "unknown capability .unknown=silly." output
++ test_i18ngrep "unknown capability .unknown=silly." output
+'
+
test_done
32: f299f46406 ! 32: 65310eaf1b setup: add support for reading extensions.objectformat
@@ Commit message
the extensions we support. If the user has specified an invalid value,
fail.
- Ensure that we don't use the value of the extension if the repository
- format version is 0.
+ Ensure that we reject the extension if the repository format version is
+ 0.
Signed-off-by: brian m. carlson <sandals@xxxxxxxxxxxxxxxxxxxx>
@@ setup.c: static int check_repo_format(const char *var, const char *value, void *
string_list_append(&data->unknown_extensions, ext);
}
-@@ setup.c: static int check_repository_format_gently(const char *gitdir, struct repository_
- repository_format_precious_objects = 0;
- set_repository_format_partial_clone(NULL);
- repository_format_worktree_config = 0;
-+ candidate->hash_algo = GIT_HASH_SHA1;
+@@ setup.c: int verify_repository_format(const struct repository_format *format,
+ return -1;
}
- string_list_clear(&candidate->unknown_extensions, 0);
+
++ if (format->version <= 0 && format->hash_algo != GIT_HASH_SHA1) {
++ strbuf_addstr(err, _("extensions.objectFormat is not valid in repo v0"));
++ return -1;
++ }
++
+ return 0;
+ }
33: 59652ad4e7 ! 33: f69cf8b1b5 Enable SHA-256 support by default
@@ t/t0001-init.sh: test_expect_success 're-init from a linked worktree' '
+ test_cmp expected actual
+'
+
-+test_expect_success 'extensions.objectFormat is not honored with repo version 0' '
++test_expect_success 'extensions.objectFormat is not allowed with repo version 0' '
+ git init --object-format=sha256 explicit-v0 &&
-+ test_config -C explicit-v0 core.repositoryformatversion 0 &&
-+ git -C explicit-v0 rev-parse --show-object-format >actual &&
-+ echo sha1 >expected &&
-+ test_cmp expected actual
++ git -C explicit-v0 config core.repositoryformatversion 0 &&
++ test_must_fail git -C explicit-v0 rev-parse --show-object-format
+'
+
+test_expect_success 'init rejects attempts to initialize with different hash' '
34: 90d38c1f09 = 34: e41d139b78 t: add test_oid option to select hash algorithm
35: 8eb68c2895 = 35: 20dbc5ff66 t: allow testing different hash algorithms via environment
36: b0b832f3d8 = 36: 4ee7f9eec8 t: make SHA1 prerequisite depend on default hash
37: f6fdad20d2 = 37: 0080a96f7b ci: run tests with SHA-256
38: dce20fcd6e ! 38: 03e1203a08 docs: add documentation for extensions.objectFormat
@@ Documentation/config/extensions.txt (new)
@@
+extensions.objectFormat::
+ Specify the hash algorithm to use. The acceptable values are `sha1` and
-+ `sha256`. If not specified, `sha1` is assumed. As with other keys under
-+ the `extensions` hierarchy, this value is ignored and has no effect unless
-+ `core.repositoryformatversion` is 1.
++ `sha256`. If not specified, `sha1` is assumed. It is an error to specify
++ this key unless `core.repositoryformatversion` is 1.
++
+Note that this setting should only be set by linkgit:git-init[1] or
+linkgit:git-clone[1]. Trying to change it after initialization will not
39: 42d1c5b8d1 = 39: fbc015258f t: remove test_oid_init in tests
