This series moves a lot of global state into the repository struct. It applies on top of 2512f15446149235156528dafbe75930c712b29e (2.16.0) It can be found at https://github.com/stefanbeller/git/tree/object-store Motivation for this series: * Easier to reason about code when all state is stored in one place, for example for multithreading * Easier to move to processing submodules in-process instead of calling a processes for the submodule handling. The last patch demonstrates this. Usually when approaching large refactoring series, a lot of tedious review work is involved and yet it is hard to be convinced the series did everything right. When reviewing this series in particular it is hard to be convinced if any function that is converted to take the repository as an argument is fully converted or if there are any hidden dependencies that are not obvious from code review. This is why this series tries to be reviewer friendly by utilizing machine assistance as much as possible. There are 3 types of patches in this series: (A) <area>: add repository argument to <function_foo> This sort of patch is just adding the repository as an argument to that function. It assumes the given repository is `the_repository` and has a compile time check for that assertion using a preprocessor trick. As did a compile check after each commit of the series, I don't expect the review burden on these patches to be high. Review on these patches is mostly checking for formatting errors or if I sneak in malicious code -- if you're inclined to believe that. (B) <area>: allow <function_foo> to handle arbitrary repositories This sort of patch is touching code inside the given function only, usually replacing all occurrences of `the_repository` by the argument `r`. Here the review is critical: Did I miss any function that relies on state of `the_repository` ? This series was developed by converting all functions of packfile/sha1-file/commit/etc using (A); after the demo patch was possible, all patches that did (A), but not (B) were deleted. Therefore I was confident at time of writing that patch that the conversion of a function which doesn't take a repository argument is okay. (C) other patches, such as moving code around, demoing this series in the last patch This approach was chosen as I think it is review friendly, despite the huge number of patches. A weakness of this approach is the buildup of a large series, which ignores the ongoing development. Rebasing that series turned out to be ok, but merging it with confidence is an issue. This series was developed partially as a pair programming exercise with Jonathan Nieder and Brandon Williams. The idea with the #define trick to separate the two very separate issues of touching a lot of code and reasoning about its correctness (hidden deps) is mostly from Jonathan Tan. Any suggestions welcome! Thanks, Stefan Brandon Williams (4): sha1_file: allow add_to_alternates_file to handle arbitrary repositories commit: convert commit_graft_pos() to handle arbitrary repositories commit: convert register_commit_graft to handle arbitrary repositories commit: convert read_graft_file to handle arbitrary repositories Jonathan Nieder (50): object-store: move packed_git and packed_git_mru to object store pack: move prepare_packed_git_run_once to object store pack: move approximate object count to object store pack: add repository argument to install_packed_git pack: add repository argument to prepare_packed_git_one pack: add repository argument to rearrange_packed_git pack: add repository argument to prepare_packed_git_mru pack: add repository argument to prepare_packed_git pack: add repository argument to reprepare_packed_git pack: add repository argument to sha1_file_name pack: add repository argument to map_sha1_file sha1_file: allow alt_odb_usable to handle arbitrary repositories pack: allow install_packed_git to handle arbitrary repositories pack: allow rearrange_packed_git to handle arbitrary repositories pack: allow prepare_packed_git_mru to handle arbitrary repositories pack: allow prepare_packed_git_one to handle arbitrary repositories pack: allow prepare_packed_git to handle arbitrary repositories pack: allow reprepare_packed_git to handle arbitrary repositories pack: allow sha1_file_name to handle arbitrary repositories pack: allow stat_sha1_file to handle arbitrary repositories pack: allow open_sha1_file to handle arbitrary repositories pack: allow map_sha1_file_1 to handle arbitrary repositories pack: allow map_sha1_file to handle arbitrary repositories pack: allow sha1_loose_object_info to handle arbitrary repositories sha1_file: add repository argument to sha1_object_info_extended object-store: move alternates API to new alternates.h object-store: move loose object functions to new loose-object.h pack: move struct pack_window and pack_entry to packfile.h object-store: move replace_objects back to object-store object-store: move lookup_replace_object to replace-object.h pack: add repository argument to packed_object_info pack: add repository argument to find_pack_entry object: add repository argument to lookup_object object: add repository argument to grow_object_hash object: add repository argument to lookup_commit_reference_gently object: add repository argument to lookup_commit object: move grafts to object parser object: add repository argument to commit_graft_pos object: add repository argument to register_commit_graft object: add repository argument to read_graft_file object: add repository argument to prepare_commit_graft object: add repository argument to lookup_commit_graft object: add repository argument to lookup_unknown_object Rename sha1_object_info.cocci to object_store.cocci alternates: add repository argument to add_to_alternates_memory object-store: move check_sha1_signature to object-store.h object-store: add repository argument to check_sha1_signature object-store: add repository argument to read_object object-store: add repository argument to read_sha1_file_extended object: move read_object_with_reference to object.h Stefan Beller (140): repository: introduce object store field object-store: move alt_odb_list and alt_odb_tail to object store sha1_file: add repository argument to alt_odb_usable sha1_file: add repository argument to link_alt_odb_entry sha1_file: add repository argument to read_info_alternates sha1_file: add repository argument to link_alt_odb_entries sha1_file: add repository argument to stat_sha1_file sha1_file: add repository argument to open_sha1_file sha1_file: add repository argument to map_sha1_file_1 sha1_file: add repository argument to sha1_loose_object_info object-store: add repository argument to prepare_alt_odb object-store: add repository argument to foreach_alt_odb object-store: allow prepare_alt_odb to handle arbitrary repositories object-store: allow foreach_alt_odb to handle arbitrary repositories replace_object.c: rename to use dash in file name replace-object: move replace_object to object store object-store: move object access functions to object-store.h replace-object: add repository argument to do_lookup_replace_object replace-object: move replace objects prepared flag to object store replace-object: check_replace_refs is safe in multi repo environment refs: add repository argument to for_each_replace_ref refs: add repository argument to get_main_ref_store replace-object: add repository argument to register_replace_object replace-object: add repository argument to register_replace_ref replace-object: add repository argument to replace_object_pos replace-object: allow replace_object_pos to handle arbitrary repositories replace-object: allow register_replace_object to handle arbitrary repositories replace-object: add repository argument to prepare_replace_object refs: store the main ref store inside the repository struct refs: allow for_each_replace_ref to handle arbitrary repositories replace-object: allow prepare_replace_object to handle arbitrary repositories replace_object: allow do_lookup_replace_object to handle arbitrary repositories replace-object: add repository argument to lookup_replace_object repository: allow lookup_replace_object to handle arbitrary repositories object-store: add repository argument to sha1_object_info pack: add repository argument to retry_bad_packed_offset pack: add repository argument to packed_to_object_type packfile: add repository argument to read_object packfile: add repository argument to unpack_entry packfile: add repository argument to cache_or_unpack_entry pack: allow find_pack_entry to handle arbitrary repositories object-store: allow sha1_object_info to handle arbitrary repositories fetch, push: do not use submodule as alternate in has_commits check push: add test showing bad interaction of replace refs and submodules replace_object: allow register_replace_ref to handle arbitrary repositories cache.h: migrate the definition of object_id to object.h repository: introduce object parser field object: add repository argument to parse_object object: add repository argument to create_object blob: add repository argument to lookup_blob tree: add repository argument to lookup_tree tag: add repository argument to lookup_tag tag: add repository argument to parse_tag_buffer tag: add repository argument to deref_tag object: add repository argument to lookup_commit_reference commit: add repository argument to parse_commit_buffer object: allow grow_object_hash to handle arbitrary repositories object: allow create_object to handle arbitrary repositories object: allow lookup_object to handle arbitrary repositories object: allow lookup_unknown_object to handle arbitrary repositories object: add repository argument to parse_object_buffer repository: keep track of all open repositories alternates: add repository argument to add_to_alternates_file object-store: add repository argument to read_sha1_file packfile: add repository argument to has_packed_and_bad packfile: allow has_packed_and_bad to handle arbitrary repositories streaming: add repository argument to open_istream_fn streaming: add repository argument to open_istream streaming: add repository argument to istream_source streaming: allow istream_source to handle arbitrary repositories sha1_file: allow read_object to handle arbitrary repositories object-store.h: allow read_sha1_file{_extended} to handle arbitrary repositories streaming: allow open_istream_incore to handle arbitrary repositories streaming: allow open_istream_pack_non_delta to handle arbitrary repositories streaming: allow open_istream_loose to handle arbitrary repositories streaming: allow open_istream to handle arbitrary repositories alternates: convert add_to_alternates_memory to handle arbitrary repos object: add repository argument to object_as_type alloc: add repository argument to alloc_blob_node alloc: add repository argument to alloc_tree_node alloc: add repository argument to alloc_commit_node alloc: add repository argument to alloc_tag_node alloc: add repository argument to alloc_object_node alloc: add repository argument to alloc_report alloc: add repository argument to alloc_commit_index alloc: allow arbitrary repositories for alloc functions object: allow object_as_type to handle arbitrary repositories commit: allow lookup_commit to handle arbitrary repositories object: add repository argument to parse_commit_gently commit: add repository argument to parse_commit commit: add repository argument to set_commit_buffer commit: add repository argument to get_cached_commit_buffer commit: add repository argument to unuse_commit_buffer commit: add repository argument to free_commit_buffer commit: allow commit buffer functions to handle arbitrary repositories tree: allow lookup_tree to handle arbitrary repositories cache: convert get_graft_file to handle arbitrary repositories shallow: add repository argument to set_alternate_shallow_file shallow: add repository argument to register_shallow shallow: add repository argument to check_shallow_file_for_update shallow: add repository argument to is_repository_shallow migrate cached path to use the_repository shallow: migrate shallow information into the object parser commit: allow prepare_commit_graft to handle arbitrary repositories commit: allow lookup_commit_graft to handle arbitrary repositories commit: allow arbitrary repository argument for parse_commit_buffer commit: allow parse_commit_gently to handle arbitrary repository commit: add repository argument to get_merge_bases_many_0 commit: add repository argument to merge_bases_many commit: add repository argument to paint_down_to_common commit: allow parse_commit to handle arbitrary repositories commit: allow paint_down_to_common to handle arbitrary repositories commit: allow merge_bases_many to handle arbitrary repositories commit: add repository argument to remove_redundant commit: allow remove_redundant to handle arbitrary repositories commit: allow get_merge_bases_many_0 to handle arbitrary repositories commit: add repository argument to get_merge_bases commit: allow get_merge_bases to handle arbitrary repositories blob: allow lookup_blob to handle arbitrary repositories tag: allow lookup_tag to handle arbitrary repositories tag: allow parse_tag_buffer to handle arbitrary repositories object: allow parse_object_buffer to handle arbitrary repositories objects: allow check_sha1_signature to handle arbitrary repositories object: allow parse_object to handle arbitrary repositories tag: allow deref_tag to handle arbitrary repositories commit: allow lookup_commit_reference_gently to handle arbitrary repositories commit: allow lookup_commit_reference to handle arbitrary repositories commit: add repository argument to get_commit_buffer commit: add repository argument to logmsg_reencode pretty: add repository argument to format_commit_message commit: allow get_commit_buffer to handle arbitrary repositories pretty: allow logmsg_reencode to handle arbitrary repositories commit: add repository argument to in_merge_bases_many commit: add repository argument to in_merge_bases commit: allow in_merge_bases_many to handle arbitrary repositories commit: allow in_merge_bases to handle arbitrary repositories pretty: allow format_commit_message to handle arbitrary repositories submodule: add repository argument to print_submodule_summary submodule: Reorder open_submodule function submodule: use submodule repos for object lookup Makefile | 2 +- alloc.c | 48 +++-- alternates.h | 68 +++++++ apply.c | 8 +- archive-tar.c | 6 +- archive-zip.c | 8 +- archive.c | 9 +- bisect.c | 6 +- blame.c | 40 ++-- blob.c | 10 +- blob.h | 2 +- branch.c | 16 +- builtin/am.c | 14 +- builtin/blame.c | 9 +- builtin/branch.c | 11 +- builtin/cat-file.c | 28 ++- builtin/checkout.c | 12 +- builtin/clone.c | 14 +- builtin/commit-tree.c | 5 +- builtin/commit.c | 63 +++--- builtin/count-objects.c | 12 +- builtin/describe.c | 19 +- builtin/diff-tree.c | 9 +- builtin/diff.c | 7 +- builtin/difftool.c | 5 +- builtin/fast-export.c | 30 +-- builtin/fetch.c | 21 +- builtin/fmt-merge-msg.c | 19 +- builtin/fsck.c | 44 ++-- builtin/gc.c | 8 +- builtin/grep.c | 10 +- builtin/hash-object.c | 1 + builtin/index-pack.c | 26 ++- builtin/log.c | 26 ++- builtin/ls-files.c | 2 +- builtin/ls-tree.c | 4 +- builtin/merge-base.c | 11 +- builtin/merge-tree.c | 12 +- builtin/merge.c | 41 ++-- builtin/mktag.c | 10 +- builtin/mktree.c | 4 +- builtin/name-rev.c | 15 +- builtin/notes.c | 16 +- builtin/pack-objects.c | 66 +++--- builtin/pack-redundant.c | 8 +- builtin/pack-refs.c | 3 +- builtin/prune-packed.c | 1 + builtin/prune.c | 8 +- builtin/pull.c | 19 +- builtin/receive-pack.c | 14 +- builtin/reflog.c | 21 +- builtin/remote.c | 1 + builtin/replace.c | 24 ++- builtin/reset.c | 11 +- builtin/rev-list.c | 7 +- builtin/rev-parse.c | 11 +- builtin/shortlog.c | 5 +- builtin/show-branch.c | 9 +- builtin/show-ref.c | 1 + builtin/submodule--helper.c | 5 +- builtin/tag.c | 12 +- builtin/unpack-file.c | 4 +- builtin/unpack-objects.c | 15 +- builtin/verify-commit.c | 7 +- bulk-checkin.c | 3 +- bundle.c | 9 +- cache-tree.c | 4 +- cache.h | 315 ++--------------------------- combine-diff.c | 5 +- commit.c | 241 ++++++++++++---------- commit.h | 49 +++-- config.c | 2 +- contrib/coccinelle/cached_path.cocci | 44 ++++ contrib/coccinelle/object_parser.cocci | 125 ++++++++++++ contrib/coccinelle/object_store.cocci | 74 +++++++ contrib/coccinelle/packed_git.cocci | 15 ++ contrib/coccinelle/submodule_reading.cocci | 34 ++++ convert.c | 1 + diff.c | 8 +- diffcore-rename.c | 1 + dir.c | 4 +- entry.c | 5 +- environment.c | 10 +- fast-import.c | 45 +++-- fetch-pack.c | 47 +++-- fsck.c | 20 +- git.c | 2 +- grep.c | 5 +- http-backend.c | 12 +- http-push.c | 24 ++- http-walker.c | 5 +- http.c | 10 +- line-log.c | 2 +- list-objects-filter.c | 2 +- list-objects.c | 4 +- log-tree.c | 18 +- loose-object.h | 91 +++++++++ mailmap.c | 4 +- match-trees.c | 6 +- merge-blobs.c | 8 +- merge-recursive.c | 30 +-- mru.h | 1 + notes-cache.c | 9 +- notes-merge.c | 15 +- notes-utils.c | 6 +- notes.c | 12 +- object-store.h | 183 +++++++++++++++++ object.c | 106 +++++----- object.h | 82 +++++++- pack-bitmap-write.c | 7 +- pack-bitmap.c | 6 +- pack-check.c | 6 +- pack-revindex.c | 1 + pack.h | 1 + packfile.c | 149 ++++++++------ packfile.h | 41 +++- parse-options-cb.c | 2 +- path.c | 18 +- path.h | 40 +++- pretty.c | 24 ++- pretty.h | 2 +- reachable.c | 12 +- read-cache.c | 7 +- ref-filter.c | 15 +- reflog-walk.c | 3 +- refs.c | 87 ++++---- refs.h | 4 +- refs/files-backend.c | 2 +- remote-testsvn.c | 6 +- remote.c | 27 ++- replace_object.c => replace-object.c | 70 +++---- replace-object.h | 34 ++++ repository.c | 84 ++++++-- repository.h | 33 ++- rerere.c | 12 +- revision.c | 48 ++--- send-pack.c | 7 +- sequencer.c | 93 +++++---- server-info.c | 12 +- sha1_file.c | 236 +++++++++++---------- sha1_name.c | 75 ++++--- shallow.c | 113 ++++++----- streaming.c | 33 +-- streaming.h | 2 +- submodule-config.c | 3 +- submodule.c | 100 +++++---- t/helper/test-example-decorate.c | 6 +- t/helper/test-ref-store.c | 6 +- t/helper/test-revision-walking.c | 2 +- t/t5531-deep-submodule-push.sh | 16 ++ tag.c | 36 ++-- tag.h | 7 +- tmp-objdir.c | 4 +- transport.c | 5 +- tree-walk.c | 5 +- tree.c | 24 ++- tree.h | 2 +- upload-pack.c | 25 ++- walker.c | 17 +- wt-status.c | 10 +- xdiff-interface.c | 4 +- 161 files changed, 2669 insertions(+), 1596 deletions(-) create mode 100644 alternates.h create mode 100644 contrib/coccinelle/cached_path.cocci create mode 100644 contrib/coccinelle/object_parser.cocci create mode 100644 contrib/coccinelle/object_store.cocci create mode 100644 contrib/coccinelle/packed_git.cocci create mode 100644 contrib/coccinelle/submodule_reading.cocci create mode 100644 loose-object.h create mode 100644 object-store.h rename replace_object.c => replace-object.c (55%) create mode 100644 replace-object.h -- 2.15.1.433.g936d1b9894.dirty
