This series is a bit of an informal follow-up to [1], adding some more substantial optimizations and usability fixes around ref filtering/formatting. Some of the changes here affect user-facing behavior, some are internal-only, but they're all interdependent enough to warrant putting them together in one series. [1] https://lore.kernel.org/git/pull.1594.v2.git.1696888736.gitgitgadget@xxxxxxxxx/ Patch 1 changes the behavior of the '--no-sort' option in 'for-each-ref', 'tag', and 'branch'. Currently, it just removes previous sort keys and, if no further keys are specified, falls back on ascending refname sort (which, IMO, makes the name '--no-sort' somewhat misleading). Now, '--no-sort' completely disables sorting (unless subsequent '--sort' options are provided). Patches 2-7 incrementally refactor various parts of the ref filtering/formatting workflows in order to create a 'filter_and_format_refs()' function. If certain conditions are met (sorting disabled, no reachability filtering or ahead-behind formatting), ref filtering & formatting is done within a single 'for_each_fullref_in' callback. Especially in large repositories, this makes a huge difference in memory usage & runtime for certain usages of 'for-each-ref', since it's no longer writing everything to a 'struct ref_array' then repeatedly whittling down/updating its contents. Patch 8 updates the 'for-each-ref' documentation, making the '--format' description a bit less jumbled and more clearly explaining the '*' prefix (to be updated in the next patch) Patch 9 changes the dereferencing done by the '*' format prefix from a single dereference to a recursive peel. See [1] + replies for the discussion that led to this approach (as opposed to a command line option or new format specifier). [1] https://lore.kernel.org/git/ZUoWWo7IEKsiSx-C@tanuki/ Finally, patch 10 adds performance tests for 'for-each-ref', showing the effects of optimizations made throughout the series. Here are some sample results from my Ubuntu VM (test names shortened for space): Test HEAD ---------------------------------------------------------------------------- 6300.2: (loose) 4.68(0.98+3.64) 6300.3: (loose, no sort) 4.65(0.91+3.67) 6300.4: (loose, --count=1) 4.50(0.84+3.60) 6300.5: (loose, --count=1, no sort) 4.24(0.46+3.71) 6300.6: (loose, tags) 2.41(0.45+1.93) 6300.7: (loose, tags, no sort) 2.33(0.48+1.83) 6300.8: (loose, tags, dereferenced) 3.65(1.66+1.95) 6300.9: (loose, tags, dereferenced, no sort) 3.48(1.59+1.87) 6300.10: for-each-ref + cat-file (loose, tags) 4.48(2.27+2.22) 6300.12: (packed) 0.90(0.68+0.18) 6300.13: (packed, no sort) 0.71(0.55+0.06) 6300.14: (packed, --count=1) 0.77(0.52+0.16) 6300.15: (packed, --count=1, no sort) 0.03(0.01+0.02) 6300.16: (packed, tags) 0.45(0.33+0.10) 6300.17: (packed, tags, no sort) 0.39(0.33+0.03) 6300.18: (packed, tags, dereferenced) 1.83(1.67+0.10) 6300.19: (packed, tags, dereferenced, no sort) 1.42(1.28+0.08) 6300.20: for-each-ref + cat-file (packed, tags) 2.36(2.11+0.29) * Victoria Changes since V1 ================ * Restructured commit message of patch 1 for better readability * Re-added 'ref_sorting_release(sorting)' to 'ls-remote' * Dropped patch 2 so we don't commit to behavior we don't want in 'for-each-ref --omit-empty --count' * Split patch 6 into one that renames 'ref_filter_handler()' to 'filter_one()' and another that creates helper functions from existing code * Added/updated code comments in patch 7, changed ref iteration "break" return value from -1 to 1 * Added a patch to reword 'for-each-ref' documentation in anticipation of updating the description of what '*' does in the format * Removed command-line option '--full-deref' for peeling tags in '*' format fields in favor of simply cutting over from the current single dereference to recursive dereference in all cases. Updated tests to match new behavior. * Added the '--count=1' tests back to p6300 (I must have unintentionally removed them before submitting V1) Victoria Dye (10): ref-filter.c: really don't sort when using --no-sort ref-filter.h: add max_count and omit_empty to ref_format ref-filter.h: move contains caches into filter ref-filter.h: add functions for filter/format & format-only ref-filter.c: rename 'ref_filter_handler()' to 'filter_one()' ref-filter.c: refactor to create common helper functions ref-filter.c: filter & format refs in the same callback for-each-ref: clean up documentation of --format ref-filter.c: use peeled tag for '*' format fields t/perf: add perf tests for for-each-ref Documentation/git-for-each-ref.txt | 23 +-- builtin/branch.c | 42 +++-- builtin/for-each-ref.c | 39 +---- builtin/ls-remote.c | 11 +- builtin/tag.c | 32 +--- ref-filter.c | 272 ++++++++++++++++++++--------- ref-filter.h | 25 +++ t/perf/p6300-for-each-ref.sh | 87 +++++++++ t/t3200-branch.sh | 68 +++++++- t/t6300-for-each-ref.sh | 43 +++++ t/t6302-for-each-ref-filter.sh | 4 +- t/t7004-tag.sh | 45 +++++ 12 files changed, 517 insertions(+), 174 deletions(-) create mode 100755 t/perf/p6300-for-each-ref.sh base-commit: bc5204569f7db44d22477485afd52ea410d83743 Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1609%2Fvdye%2Fvdye%2Ffor-each-ref-optimizations-v2 Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1609/vdye/vdye/for-each-ref-optimizations-v2 Pull-Request: https://github.com/gitgitgadget/git/pull/1609 Range-diff vs v1: 1: dea8d7d1e86 ! 1: 074da1ff3e8 ref-filter.c: really don't sort when using --no-sort @@ Metadata ## Commit message ## ref-filter.c: really don't sort when using --no-sort - Update 'ref_sorting_options()' to return a NULL 'struct ref_sorting *' if - the string list provided to it is empty, rather than returning the default - refname sort structure. Also update 'ref_array_sort()' to explicitly skip - sorting if its 'struct ref_sorting *' arg is NULL. Other functions using - 'struct ref_sorting *' do not need any changes because they already properly - ignore NULL values. + When '--no-sort' is passed to 'for-each-ref', 'tag', and 'branch', the + printed refs are still sorted by ascending refname. Change the handling of + sort options in these commands so that '--no-sort' to truly disables + sorting. + + '--no-sort' does not disable sorting in these commands is because their + option parsing does not distinguish between "the absence of '--sort'" + (and/or values for tag.sort & branch.sort) and '--no-sort'. Both result in + an empty 'sorting_options' string list, which is parsed by + 'ref_sorting_options()' to create the 'struct ref_sorting *' for the + command. If the string list is empty, 'ref_sorting_options()' interprets + that as "the absence of '--sort'" and returns the default ref sorting + structure (equivalent to "refname" sort). - The goal of this change is to have the '--no-sort' option truly disable - sorting in commands like 'for-each-ref, 'tag', and 'branch'. Right now, - '--no-sort' will still trigger refname sorting by default in 'for-each-ref', - 'tag', and 'branch'. + To handle '--no-sort' properly while preserving the "refname" sort in the + "absence of --sort'" case, first explicitly add "refname" to the string list + *before* parsing options. This alone doesn't actually change any behavior, + since 'compare_refs()' already falls back on comparing refnames if two refs + are equal w.r.t all other sort keys. - To match existing behavior as closely as possible, explicitly add "refname" - to the list of sort keys in 'for-each-ref', 'tag', and 'branch' before - parsing options (if no config-based sort keys are set). This ensures that - sorting will only be fully disabled if '--no-sort' is provided as an option; - otherwise, "refname" sorting will remain the default. Note: this also means - that even when sort keys are provided on the command line, "refname" will be - the final sort key in the sorting structure. This doesn't actually change - any behavior, since 'compare_refs()' already falls back on comparing - refnames if two refs are equal w.r.t all other sort keys. + Now that the string list is populated by default, '--no-sort' is the only + way to empty the 'sorting_options' string list. Update + 'ref_sorting_options()' to return a NULL 'struct ref_sorting *' if the + string list is empty, and add a condition to 'ref_array_sort()' to skip the + sort altogether if the sort structure is NULL. Note that other functions + using 'struct ref_sorting *' do not need any changes because they already + ignore NULL values. Finally, remove the condition around sorting in 'ls-remote', since it's no - longer necessary. Unlike 'for-each-ref' et. al., it does *not* set any sort - keys by default. The default empty list of sort keys will produce a NULL - 'struct ref_sorting *', which causes the sorting to be skipped in - 'ref_array_sort()'. + longer necessary. Unlike 'for-each-ref' et. al., it does *not* do any + sorting by default. This default is preserved by simply leaving its sort key + string list empty before parsing options; if no additional sort keys are + set, 'struct ref_sorting *' is NULL and sorting is skipped. Signed-off-by: Victoria Dye <vdye@xxxxxxxxxx> @@ builtin/ls-remote.c: int cmd_ls_remote(int argc, const char **argv, const char * for (i = 0; i < ref_array.nr; i++) { const struct ref_array_item *ref = ref_array.items[i]; +@@ builtin/ls-remote.c: int cmd_ls_remote(int argc, const char **argv, const char *prefix) + status = 0; /* we found something */ + } + ++ ref_sorting_release(sorting); + ref_array_clear(&ref_array); + if (transport_disconnect(transport)) + status = 1; ## builtin/tag.c ## @@ builtin/tag.c: int cmd_tag(int argc, const char **argv, const char *prefix) 2: 88eba4146cd < -: ----------- for-each-ref: clarify interaction of --omit-empty & --count 3: 2e2f9738205 = 2: adac101bc60 ref-filter.h: add max_count and omit_empty to ref_format 4: 6c66445ee31 ! 3: f44c4b42c93 ref-filter.h: move contains caches into filter @@ Commit message filter struct will support, so they are updated to be internally accessible wherever the filter is used. - The design used here is mirrors what was introduced in 576de3d956 + The design used here mirrors what was introduced in 576de3d956 (unpack_trees: start splitting internal fields from public API, 2023-02-27) for 'unpack_trees_options'. 5: f5be57eea7d = 4: 187b1d6610f ref-filter.h: add functions for filter/format & format-only -: ----------- > 5: 040d291ca45 ref-filter.c: rename 'ref_filter_handler()' to 'filter_one()' 6: 8c77452e5dd ! 6: 633c0c74c2e ref-filter.c: refactor to create common helper functions @@ Commit message ref-filter.c: refactor to create common helper functions Factor out parts of 'ref_array_push()', 'ref_filter_handler()', and - 'filter_refs()' into new helper functions ('ref_array_append()', - 'apply_ref_filter()', and 'do_filter_refs()' respectively), as well as - rename 'ref_filter_handler()' to 'filter_one()'. In this and later - patches, these helpers will be used by new ref-filter API functions. This - patch does not result in any user-facing behavior changes or changes to - callers outside of 'ref-filter.c'. + 'filter_refs()' into new helper functions: - The changes are as follows: + * Extract the code to grow a 'struct ref_array' and append a given 'struct + ref_array_item *' to it from 'ref_array_push()' into 'ref_array_append()'. + * Extract the code to filter a given ref by refname & object ID then create + a new 'struct ref_array_item *' from 'filter_one()' into + 'apply_ref_filter()'. + * Extract the code for filter pre-processing, contains cache creation, and + ref iteration from 'filter_refs()' into 'do_filter_refs()'. - * The logic to grow a 'struct ref_array' and append a given 'struct - ref_array_item *' to it is extracted from 'ref_array_push()' into - 'ref_array_append()'. - * 'ref_filter_handler()' is renamed to 'filter_one()' to more clearly - distinguish it from other ref filtering callbacks that will be added in - later patches. The "*_one()" naming convention is common throughout the - codebase for iteration callbacks. - * The code to filter a given ref by refname & object ID then create a new - 'struct ref_array_item' is moved out of 'filter_one()' and into - 'apply_ref_filter()'. 'apply_ref_filter()' returns either NULL (if the ref - does not match the given filter) or a 'struct ref_array_item *' created - with 'new_ref_array_item()'; 'filter_one()' appends that item to - its ref array with 'ref_array_append()'. - * The filter pre-processing, contains cache creation, and ref iteration of - 'filter_refs()' is extracted into 'do_filter_refs()'. 'do_filter_refs()' - takes its ref iterator function & callback data as an input from the - caller, setting it up to be used with additional filtering callbacks in - later patches. + In later patches, these helpers will be used by new ref-filter API + functions. This patch does not result in any user-facing behavior changes or + changes to callers outside of 'ref-filter.c'. Signed-off-by: Victoria Dye <vdye@xxxxxxxxxx> @@ ref-filter.c: static int filter_ref_kind(struct ref_filter *filter, const char * - * A call-back given to for_each_ref(). Filter refs and keep them for - * later object processing. - */ --static int ref_filter_handler(const char *refname, const struct object_id *oid, int flag, void *cb_data) +-static int filter_one(const char *refname, const struct object_id *oid, int flag, void *cb_data) +static struct ref_array_item *apply_ref_filter(const char *refname, const struct object_id *oid, + int flag, struct ref_filter *filter) { @@ ref-filter.c: static int filter_ref_kind(struct ref_filter *filter, const char * /* * A merge filter is applied on refs pointing to commits. Hence -@@ ref-filter.c: static int ref_filter_handler(const char *refname, const struct object_id *oid, +@@ ref-filter.c: static int filter_one(const char *refname, const struct object_id *oid, int flag filter->with_commit || filter->no_commit || filter->verbose) { commit = lookup_commit_reference_gently(the_repository, oid, 1); if (!commit) @@ ref-filter.c: static int ref_filter_handler(const char *refname, const struct ob } /* -@@ ref-filter.c: static int ref_filter_handler(const char *refname, const struct object_id *oid, +@@ ref-filter.c: static int filter_one(const char *refname, const struct object_id *oid, int flag * to do its job and the resulting list may yet to be pruned * by maxcount logic. */ @@ ref-filter.c: int filter_refs(struct ref_array *array, struct ref_filter *filter * of filter_ref_kind(). */ if (filter->kind == FILTER_REFS_BRANCHES) -- ret = for_each_fullref_in("refs/heads/", ref_filter_handler, &ref_cbdata); +- ret = for_each_fullref_in("refs/heads/", filter_one, &ref_cbdata); + ret = for_each_fullref_in("refs/heads/", fn, cb_data); else if (filter->kind == FILTER_REFS_REMOTES) -- ret = for_each_fullref_in("refs/remotes/", ref_filter_handler, &ref_cbdata); +- ret = for_each_fullref_in("refs/remotes/", filter_one, &ref_cbdata); + ret = for_each_fullref_in("refs/remotes/", fn, cb_data); else if (filter->kind == FILTER_REFS_TAGS) -- ret = for_each_fullref_in("refs/tags/", ref_filter_handler, &ref_cbdata); +- ret = for_each_fullref_in("refs/tags/", filter_one, &ref_cbdata); + ret = for_each_fullref_in("refs/tags/", fn, cb_data); else if (filter->kind & FILTER_REFS_ALL) -- ret = for_each_fullref_in_pattern(filter, ref_filter_handler, &ref_cbdata); +- ret = for_each_fullref_in_pattern(filter, filter_one, &ref_cbdata); + ret = for_each_fullref_in_pattern(filter, fn, cb_data); if (!ret && (filter->kind & FILTER_REFS_DETACHED_HEAD)) -- head_ref(ref_filter_handler, &ref_cbdata); +- head_ref(filter_one, &ref_cbdata); + head_ref(fn, cb_data); } 7: 84db440896c ! 7: 91a77c1a834 ref-filter.c: filter & format refs in the same callback @@ ref-filter.c: static void free_array_item(struct ref_array_item *item) + strbuf_release(&err); + free_array_item(ref); + ++ /* ++ * Increment the running count of refs that match the filter. If ++ * max_count is set and we've reached the max, stop the ref ++ * iteration by returning a nonzero value. ++ */ + if (ref_cbdata->format->array_opts.max_count && + ++ref_cbdata->internal.count >= ref_cbdata->format->array_opts.max_count) -+ return -1; ++ return 1; + + return 0; +} @@ ref-filter.c: int filter_refs(struct ref_array *array, struct ref_filter *filter + struct ref_format *format) +{ + /* -+ * Refs can be filtered and formatted in the same iteration as long -+ * as we aren't filtering on reachability, sorting the results, or -+ * including ahead-behind information in the formatted output. ++ * Filtering & formatting results within a single ref iteration ++ * callback is not compatible with options that require ++ * post-processing a filtered ref_array. These include: ++ * - filtering on reachability ++ * - sorting the filtered results ++ * - including ahead-behind information in the formatted output + */ + return !(filter->reachable_from || + filter->unreachable_from || -: ----------- > 8: 8eb2fc2950c for-each-ref: clean up documentation of --format 8: 352b5c42ac3 ! 9: 48254d8e161 for-each-ref: add option to fully dereference tags @@ Metadata Author: Victoria Dye <vdye@xxxxxxxxxx> ## Commit message ## - for-each-ref: add option to fully dereference tags + ref-filter.c: use peeled tag for '*' format fields - Add a boolean flag '--full-deref' that, when enabled, fills '%(*fieldname)' - format fields using the fully peeled target of tag objects, rather than the - immediate target. - - In other builtins ('rev-parse', 'show-ref'), "dereferencing" tags typically - means peeling them down to their non-tag target. Unlike these commands, - 'for-each-ref' dereferences only one "level" of tags in '*' format fields - (like "%(*objectname)"). For most annotated tags, one level of dereferencing - is enough, since most tags point to commits or trees. However, nested tags - (annotated tags whose target is another annotated tag) dereferenced once - will point to their target tag, different a full peel to e.g. a commit. + In most builtins ('rev-parse <revision>^{}', 'show-ref --dereference'), + "dereferencing" a tag refers to a recursive peel of the tag object. Unlike + these cases, the dereferencing prefix ('*') in 'for-each-ref' format + specifiers triggers only a single, non-recursive dereference of a given tag + object. For most annotated tags, a single dereference is all that is needed + to access the tag's associated commit or tree; "recursive" and + "non-recursive" dereferencing are functionally equivalent in these cases. + However, nested tags (annotated tags whose target is another annotated tag) + dereferenced once return another tag, where a recursive dereference would + return the commit or tree. Currently, if a user wants to filter & format refs and include information - about the fully dereferenced tag, they can do so with something like + about a recursively-dereferenced tag, they can do so with something like 'cat-file --batch-check': git for-each-ref --format="%(objectname)^{} %(refname)" <pattern> | git cat-file --batch-check="%(objectname) %(rest)" But the combination of commands is inefficient. So, to improve the - efficiency of this use case, add a '--full-deref' option that causes - 'for-each-ref' to fully dereference tags when formatting with '*' fields. + performance of this use case and align the defererencing behavior of + 'for-each-ref' with that of other commands, update the ref formatting code + to use the peeled tag (from 'peel_iterated_oid()') to populate '*' fields + rather than the tag's immediate target object (from 'get_tagged_oid()'). + + Additionally, add a test to 't6300-for-each-ref' to verify new nested tag + behavior and update 't6302-for-each-ref-filter.sh' to print the correct + value for nested dereferenced fields. Signed-off-by: Victoria Dye <vdye@xxxxxxxxxx> ## Documentation/git-for-each-ref.txt ## -@@ Documentation/git-for-each-ref.txt: SYNOPSIS - 'git for-each-ref' [--count=<count>] [--shell|--perl|--python|--tcl] - [(--sort=<key>)...] [--format=<format>] - [ --stdin | <pattern>... ] -+ [--full-deref] - [--points-at=<object>] - [--merged[=<object>]] [--no-merged[=<object>]] - [--contains[=<object>]] [--no-contains[=<object>]] -@@ Documentation/git-for-each-ref.txt: OPTIONS - the specified host language. This is meant to produce - a scriptlet that can directly be `eval`ed. +@@ Documentation/git-for-each-ref.txt: from the `committer` or `tagger` fields depending on the object type. + These are intended for working on a mix of annotated and lightweight tags. -+--full-deref:: -+ Populate dereferenced format fields (indicated with an asterisk (`*`) -+ prefix before the fieldname) with information about the fully-peeled -+ target object of a tag ref, rather than its immediate target object. -+ This only affects the output for nested annotated tags, where the tag's -+ immediate target is another tag but its fully-peeled target is another -+ object type (e.g. a commit). -+ - --points-at=<object>:: - Only list refs which points at the given object. + For tag objects, a `fieldname` prefixed with an asterisk (`*`) expands to +-the `fieldname` value of object the tag points at, rather than that of the +-tag object itself. ++the `fieldname` value of the peeled object, rather than that of the tag ++object itself. - - ## builtin/for-each-ref.c ## -@@ builtin/for-each-ref.c: int cmd_for_each_ref(int argc, const char **argv, const char *prefix) - OPT_INTEGER( 0 , "count", &format.array_opts.max_count, N_("show only <n> matched refs")), - OPT_STRING( 0 , "format", &format.format, N_("format"), N_("format to use for the output")), - OPT__COLOR(&format.use_color, N_("respect format colors")), -+ OPT_BOOL(0, "full-deref", &format.full_deref, -+ N_("fully dereference tags to populate '*' format fields")), - OPT_REF_FILTER_EXCLUDE(&filter), - OPT_REF_SORT(&sorting_options), - OPT_CALLBACK(0, "points-at", &filter.points_at, + Fields that have name-email-date tuple as its value (`author`, + `committer`, and `tagger`) can be suffixed with `name`, `email`, ## ref-filter.c ## -@@ ref-filter.c: static struct used_atom { - char *head; - } u; - } *used_atom; --static int used_atom_cnt, need_tagged, need_symref; -+static int used_atom_cnt, need_symref; -+ -+enum tag_dereference_mode { -+ NO_DEREF = 0, -+ DEREF_ONE, -+ DEREF_ALL -+}; -+static enum tag_dereference_mode need_tagged; - - /* - * Expand string, append it to strbuf *sb, then return error code ret. -@@ ref-filter.c: static int parse_ref_filter_atom(struct ref_format *format, - memset(&used_atom[at].u, 0, sizeof(used_atom[at].u)); - if (valid_atom[i].parser && valid_atom[i].parser(format, &used_atom[at], arg, err)) - return -1; -- if (*atom == '*') -- need_tagged = 1; -+ if (*atom == '*' && !need_tagged) -+ need_tagged = format->full_deref ? DEREF_ALL : DEREF_ONE; - if (i == ATOM_SYMREF) - need_symref = 1; - return at; @@ ref-filter.c: static int populate_value(struct ref_array_item *ref, struct strbuf *err) - * If it is a tag object, see if we use a value that derefs - * the object, and if we do grab the object it refers to. + return 0; + + /* +- * If it is a tag object, see if we use a value that derefs +- * the object, and if we do grab the object it refers to. ++ * If it is a tag object, see if we use the peeled value. If we do, ++ * grab the peeled OID. */ - oi_deref.oid = *get_tagged_oid((struct tag *)obj); -+ if (need_tagged == DEREF_ALL) { -+ if (peel_iterated_oid(&obj->oid, &oi_deref.oid)) -+ die("bad tag"); -+ } else { -+ oi_deref.oid = *get_tagged_oid((struct tag *)obj); -+ } ++ if (need_tagged && peel_iterated_oid(&obj->oid, &oi_deref.oid)) ++ die("bad tag"); - /* - * NEEDSWORK: This derefs tag only once, which @@ ref-filter.c: static int populate_value(struct ref_array_item *ref, struct strbu } - ## ref-filter.h ## -@@ ref-filter.h: struct ref_format { - const char *rest; - int quote_style; - int use_color; -+ int full_deref; - - /* Internal state to ref-filter */ - int need_color_reset_at_eol; - ## t/t6300-for-each-ref.sh ## @@ t/t6300-for-each-ref.sh: test_expect_success 'git for-each-ref with non-existing refs' ' test_must_be_empty actual @@ t/t6300-for-each-ref.sh: test_expect_success 'git for-each-ref with non-existing + nest1_tag_oid="$(git rev-parse refs/tags/nested/nest1)" && + nest2_tag_oid="$(git rev-parse refs/tags/nested/nest2)" && + -+ # Without full dereference -+ cat >expect <<-EOF && -+ refs/tags/nested/base $base_tag_oid tag $head_oid commit -+ refs/tags/nested/nest1 $nest1_tag_oid tag $base_tag_oid tag -+ refs/tags/nested/nest2 $nest2_tag_oid tag $nest1_tag_oid tag -+ EOF -+ -+ git for-each-ref --format="%(refname) %(objectname) %(objecttype) %(*objectname) %(*objecttype)" \ -+ refs/tags/nested/ >actual && -+ test_cmp expect actual && -+ -+ # With full dereference + cat >expect <<-EOF && + refs/tags/nested/base $base_tag_oid tag $head_oid commit + refs/tags/nested/nest1 $nest1_tag_oid tag $head_oid commit + refs/tags/nested/nest2 $nest2_tag_oid tag $head_oid commit + EOF + -+ git for-each-ref --full-deref \ ++ git for-each-ref \ + --format="%(refname) %(objectname) %(objecttype) %(*objectname) %(*objecttype)" \ + refs/tags/nested/ >actual && + test_cmp expect actual @@ t/t6300-for-each-ref.sh: test_expect_success 'git for-each-ref with non-existing GRADE_FORMAT="%(signature:grade)%0a%(signature:key)%0a%(signature:signer)%0a%(signature:fingerprint)%0a%(signature:primarykeyfingerprint)" TRUSTLEVEL_FORMAT="%(signature:trustlevel)%0a%(signature:key)%0a%(signature:signer)%0a%(signature:fingerprint)%0a%(signature:primarykeyfingerprint)" + + ## t/t6302-for-each-ref-filter.sh ## +@@ t/t6302-for-each-ref-filter.sh: test_expect_success 'check signed tags with --points-at' ' + sed -e "s/Z$//" >expect <<-\EOF && + refs/heads/side Z + refs/tags/annotated-tag four +- refs/tags/doubly-annotated-tag An annotated tag +- refs/tags/doubly-signed-tag A signed tag ++ refs/tags/doubly-annotated-tag four ++ refs/tags/doubly-signed-tag four + refs/tags/four Z + refs/tags/signed-tag four + EOF 9: a409d773057 ! 10: d51d073aa4a t/perf: add perf tests for for-each-ref @@ Commit message Add performance tests for 'for-each-ref'. The tests exercise different combinations of filters/formats/options, as well as the overall performance of 'git for-each-ref | git cat-file --batch-check' to demonstrate the - performance difference vs. 'git for-each-ref --full-deref'. + performance difference vs. 'git for-each-ref' with "%(*fieldname)" format + specifiers. All tests are run against a repository with 40k loose refs - 10k commits, each having a unique: @@ t/perf/p6300-for-each-ref.sh (new) +run_tests () { + test_for_each_ref "$1" + test_for_each_ref "$1, no sort" --no-sort ++ test_for_each_ref "$1, --count=1" --count=1 ++ test_for_each_ref "$1, --count=1, no sort" --no-sort --count=1 + test_for_each_ref "$1, tags" refs/tags/ + test_for_each_ref "$1, tags, no sort" --no-sort refs/tags/ -+ test_for_each_ref "$1, tags, shallow deref" '--format="%(refname) %(objectname) %(*objectname)"' refs/tags/ -+ test_for_each_ref "$1, tags, shallow deref, no sort" --no-sort '--format="%(refname) %(objectname) %(*objectname)"' refs/tags/ -+ test_for_each_ref "$1, tags, full deref" --full-deref '--format="%(refname) %(objectname) %(*objectname)"' refs/tags/ -+ test_for_each_ref "$1, tags, full deref, no sort" --no-sort --full-deref '--format="%(refname) %(objectname) %(*objectname)"' refs/tags/ ++ test_for_each_ref "$1, tags, dereferenced" '--format="%(refname) %(objectname) %(*objectname)"' refs/tags/ ++ test_for_each_ref "$1, tags, dereferenced, no sort" --no-sort '--format="%(refname) %(objectname) %(*objectname)"' refs/tags/ + -+ test_perf "for-each-ref ($1, tags) + cat-file --batch-check (full deref)" " ++ test_perf "for-each-ref ($1, tags) + cat-file --batch-check (dereferenced)" " + for i in \$(test_seq $test_iteration_count); do + git for-each-ref --format='%(objectname)^{} %(refname) %(objectname)' refs/tags/ | \ + git cat-file --batch-check='%(objectname) %(rest)' >/dev/null -- gitgitgadget
