We manage some internal repositories with numerous CI tasks, each requiring code preparation through git-clone or git-fetch. These tasks, triggered by post-receive hooks, often fetch the same copy of code concurrently using --depth=1, causing extremely high load spikes on our Git servers. To reduce performance impacts caused by these tasks, we plan to deploy a specially designed pack-objects-hook [1]. This hook would allow the packs generated by git-pack-objects(during git-clone or git-fetch) to be reused. Since not all clone/fetch operations will benefit from this caching (e.g., pulls from developer environments), clients need to pass a special identifier to indicate whether caching should be enabled. Using server options [2] is suitable for this purpose. However, server options can only be specified via the command line option (via --server-option or -o), which is inconvenient and requires modifications to CI scripts. A configuration-based approach is preferable, as it can be propagated through global configuration (e.g. ~/.gitconfig) and avoids compatibility issues with older Git versions that don't support --server-option. This patch series introduces a new multi-valued configuration, remote.<name>.serverOption, similar to push.pushOption, to specify default server options for the corresponding remote. * Patches 1~3 contain the main changes for introducing the new configuration. * Patch 4 fixes a issue for git-fetch not sending server-options when fetching from multiple remotes. * Patch 5 is a minor fix for a server options-related memory leak. 1. https://git-scm.com/docs/git-config#Documentation/git-config.txt-uploadpackpackObjectsHook 2. https://git-scm.com/docs/gitprotocol-v2#_server_option Xing Xin (5): transport: introduce parse_transport_option() method remote: introduce remote.<name>.serverOption configuration transport.c::handshake: make use of server options from remote fetch: respect --server-option when fetching multiple remotes ls-remote: leakfix for not clearing server_options Documentation/config/remote.txt | 10 +++ Documentation/fetch-options.txt | 3 + Documentation/git-clone.txt | 3 + Documentation/git-ls-remote.txt | 3 + builtin/fetch.c | 2 + builtin/ls-remote.c | 1 + builtin/push.c | 9 +-- remote.c | 6 ++ remote.h | 3 + t/t5702-protocol-v2.sh | 133 ++++++++++++++++++++++++++++++++ transport.c | 15 ++++ transport.h | 4 + 12 files changed, 184 insertions(+), 8 deletions(-) base-commit: 6258f68c3c1092c901337895c864073dcdea9213 Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1776%2Fblanet%2Fxx%2Fadd-server-option-from-config-v3 Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1776/blanet/xx/add-server-option-from-config-v3 Pull-Request: https://github.com/git/git/pull/1776 Range-diff vs v2: 1: c95ed5e0dd5 = 1: b44face42e1 transport: introduce parse_transport_option() method 2: 2474b4c69d6 ! 2: 3c6b129d368 remote: introduce remote.<name>.serverOption configuration @@ Documentation/config/remote.txt: remote.<name>.partialclonefilter:: database, use the `--refetch` option of linkgit:git-fetch[1]. + +remote.<name>.serverOption:: -+ When no `--server-option=<option>` argument is given from the command -+ line, git will use the values from this configuration as a default list of -+ server options for this remote. ++ The default set of server options used when fetching from this remote. ++ These server options can be overridden by the `--server-option=` command ++ line arguments. ++ +This is a multi-valued variable, and an empty value can be used in a higher +priority configuration file (e.g. `.git/config` in a repository) to clear @@ remote.c enum map_direction { FROM_SRC, FROM_DST }; -@@ remote.c: static struct remote *make_remote(struct remote_state *remote_state, - struct remote *ret; - struct remotes_hash_key lookup; - struct hashmap_entry lookup_entry, *e; -+ struct string_list server_options = STRING_LIST_INIT_DUP; - - if (!len) - len = strlen(name); @@ remote.c: static struct remote *make_remote(struct remote_state *remote_state, ret->name = xstrndup(name, len); refspec_init(&ret->push, REFSPEC_PUSH); refspec_init(&ret->fetch, REFSPEC_FETCH); -+ ret->server_options = server_options; ++ string_list_init_dup(&ret->server_options); ALLOC_GROW(remote_state->remotes, remote_state->remotes_nr + 1, remote_state->remotes_alloc); @@ remote.c: static void remote_clear(struct remote *remote) static void add_merge(struct branch *branch, const char *name) @@ remote.c: static int handle_config(const char *key, const char *value, - key, value); } else if (!strcmp(subkey, "vcs")) { + FREE_AND_NULL(remote->foreign_vcs); return git_config_string(&remote->foreign_vcs, key, value); + } else if (!strcmp(subkey, "serveroption")) { + return parse_transport_option(key, value, 3: a7f3e458501 = 3: f0835259b06 transport.c::handshake: make use of server options from remote 4: 39ee8dbef78 = 4: 420b15d9f37 fetch: respect --server-option when fetching multiple remotes 5: 39c07a6c8ee ! 5: 2528d929c7e ls-remote: leakfix for not clearing server_options @@ Commit message Signed-off-by: Xing Xin <xingxin.xx@xxxxxxxxxxxxx> ## builtin/ls-remote.c ## -@@ builtin/ls-remote.c: int cmd_ls_remote(int argc, const char **argv, const char *prefix) +@@ builtin/ls-remote.c: int cmd_ls_remote(int argc, transport_ls_refs_options_release(&transport_options); strvec_clear(&pattern); -- gitgitgadget
