This is part 2 of 3 of the SHA-256 work. This series, which is
unfortunately longer than I'd like, adds all of the protocol logic to
work with SHA-256 repositories.
It was originally planned that we would not upgrade the protocol and
would use SHA-1 for all protocol functionality until some point in the
future. However, doing that requires a huge amount of additional work
(probably incorporating several hundred more patches which are not yet
written) and it's not possible to get the test suite to even come close
to passing without a way to fetch and push repositories. I therefore
decided that implementing an object-format extension was the best way
forward.
This series provides object-format extensions for both the original and
v2 protocols, including full documentation. Helpers, such as
git-remote-https, also learn capabilities to pass the object-format
extension back and forth, and to query its state. The code is designed
to allow multiple object-format extensions to be provided if the server
supports multiple algorithms for one repo and to default to SHA-1 if no
object-format extension is provided.
The other two cases are the dumb HTTP protocol and bundles, both of
which have no object-format extension (because they provide no
capabilities) and are therefore distinguished solely by their hash
length. We will have problems if in the future we need to use another
256-bit algorithm, but I plan to be improvident and hope that we'll move
to longer algorithms in the future to cover ourselves for post-quantum
security.
Clone support is necessarily a little tricky because we are initializing
a repository and then fetching refs, at which point we learn what hash
algorithm the remote side supports. We work around this by calling the
code that updates the hash algorithm and repository version a second
time to rewrite that data once we know what version we're using. This
is the most robust way I could approach this problem, but it is still a
little ugly.
As mentioned, this series is longer than I'd like, but it is complete:
this is all the SHA-256 protocol work. Additional future series include
one last series of test fixes (28 patches) plus six final patches in the
series that enables SHA-256 support.
brian m. carlson (44):
t1050: match object ID paths in a hash-insensitive way
Documentation: document v1 protocol object-format capability
connect: have ref processing code take struct packet_reader
wrapper: add function to compare strings with different NUL
termination
remote: advertise the object-format capability on the server side
connect: add function to parse multiple v1 capability values
connect: add function to fetch value of a v2 server capability
pkt-line: add a member for hash algorithm
transport: add a hash algorithm member
connect: add function to detect supported v1 hash functions
send-pack: detect when the server doesn't support our hash
connect: make parse_feature_value extern
fetch-pack: detect when the server doesn't support our hash
connect: detect algorithm when fetching refs
builtin/receive-pack: detect when the server doesn't support our hash
docs: update remote helper docs for object-format extensions
transport-helper: implement object-format extensions
remote-curl: implement object-format extensions
builtin/clone: initialize hash algorithm properly
t5562: pass object-format in synthesized test data
t5704: send object-format capability with SHA-256
fetch-pack: parse and advertise the object-format capability
setup: set the_repository's hash algo when checking format
t3200: mark assertion with SHA1 prerequisite
packfile: compute and use the index CRC offset
t5302: modernize test formatting
builtin/show-index: provide options to determine hash algo
t1302: expect repo format version 1 for SHA-256
Documentation/technical: document object-format for protocol v2
connect: pass full packet reader when parsing v2 refs
connect: parse v2 refs with correct hash algorithm
serve: advertise object-format capability for protocol v2
t5500: make hash independent
builtin/ls-remote: initialize repository based on fetch
remote-curl: detect algorithm for dumb HTTP by size
builtin/index-pack: add option to specify hash algorithm
t1050: pass algorithm to index-pack when outside repo
remote-curl: avoid truncating refs with ls-remote
t/helper: initialize the repository for test-sha1-array
t5702: offer an object-format capability in the test
t5703: use object-format serve option
t5300: pass --object-format to git index-pack
bundle: detect hash algorithm when reading refs
remote-testgit: adapt for object-format
Documentation/gitremote-helpers.txt | 33 +-
.../technical/protocol-capabilities.txt | 16 +-
Documentation/technical/protocol-v2.txt | 9 +
builtin/clone.c | 9 +
builtin/index-pack.c | 11 +-
builtin/ls-remote.c | 4 +
builtin/receive-pack.c | 10 +
builtin/show-index.c | 29 +-
bundle.c | 22 +-
bundle.h | 1 +
connect.c | 136 +++++--
connect.h | 3 +
fetch-pack.c | 14 +
git-compat-util.h | 2 +
git.c | 2 +-
object-store.h | 1 +
packfile.c | 1 +
pkt-line.c | 1 +
pkt-line.h | 3 +
remote-curl.c | 46 ++-
send-pack.c | 6 +
serve.c | 27 ++
setup.c | 1 +
t/helper/test-oid-array.c | 3 +
t/t1050-large.sh | 6 +-
t/t1302-repo-version.sh | 6 +-
t/t3200-branch.sh | 2 +-
t/t5300-pack-object.sh | 9 +-
t/t5302-pack-index.sh | 360 +++++++++---------
t/t5500-fetch-pack.sh | 5 +-
t/t5562-http-backend-content-length.sh | 14 +-
t/t5701-git-serve.sh | 28 +-
t/t5702-protocol-v2.sh | 2 +
t/t5703-upload-pack-ref-in-want.sh | 19 +-
t/t5704-protocol-violations.sh | 12 +
t/t5801/git-remote-testgit | 6 +
t/test-lib.sh | 1 +
transport-helper.c | 24 +-
transport.c | 18 +-
transport.h | 8 +
upload-pack.c | 3 +-
wrapper.c | 12 +
42 files changed, 670 insertions(+), 255 deletions(-)
