Hi,
this is the third version of my patch series to speed up mirror-fetches
with many refs. This patch series applies on top of master with
ps/connectivity-optim merged into it.
There's only some smallish changes based on Stolee's feedback (thanks
for that!):
- A small typo in 1/7.
- A confict fix in 4/7 required now because it's based on master
instead of directly on my merged topic.
- I've adjusted patch 5/7 such that I don't have to re-touch the
logic in 6/7.
Patrick
Patrick Steinhardt (7):
fetch: speed up lookup of want refs via commit-graph
fetch: avoid unpacking headers in object existence check
connected: refactor iterator to return next object ID directly
fetch-pack: optimize loading of refs via commit graph
fetch: refactor fetch refs to be more extendable
fetch: merge fetching and consuming refs
fetch: avoid second connectivity check if we already have all objects
builtin/clone.c | 8 ++---
builtin/fetch.c | 74 +++++++++++++++++++++++-------------------
builtin/receive-pack.c | 17 ++++------
connected.c | 15 +++++----
connected.h | 2 +-
fetch-pack.c | 12 ++++---
6 files changed, 67 insertions(+), 61 deletions(-)
Range-diff against v2:
1: 4a819a6830 ! 1: 8214f04971 fetch: speed up lookup of want refs via commit-graph
@@ Commit message
that we repeatedly need to unpack object headers for each of the
referenced objects.
- Speed this up by opportunistcally trying to resolve object IDs via the
+ Speed this up by opportunistically trying to resolve object IDs via the
commit graph. We only do so for any refs which are not in "refs/tags":
more likely than not, these are going to be a commit anyway, and this
lets us avoid having to unpack object headers completely in case the
2: 81ebadabe8 = 2: 991a27cb82 fetch: avoid unpacking headers in object existence check
3: 98e981ced9 = 3: ba834803ab connected: refactor iterator to return next object ID directly
4: 6311203f08 ! 4: 99d3316d48 fetch-pack: optimize loading of refs via commit graph
@@ fetch-pack.c: static struct commit *deref_without_lazy_fetch(const struct object
while (1) {
if (oid_object_info_extended(the_repository, oid, &info,
-@@ fetch-pack.c: static struct commit *deref_without_lazy_fetch(const struct object_id *oid,
- }
-
- if (type == OBJ_COMMIT) {
-- struct commit *commit = lookup_commit(the_repository, oid);
-+ commit = lookup_commit(the_repository, oid);
- if (!commit || repo_parse_commit(the_repository, commit))
- return NULL;
- return commit;
5: 56a9158ac3 ! 5: d64888e072 fetch: refactor fetch refs to be more extendable
@@ builtin/fetch.c: static int check_exist_and_connected(struct ref *ref_map)
trace2_region_enter("fetch", "fetch_refs", the_repository);
ret = transport_fetch_refs(transport, ref_map);
trace2_region_leave("fetch", "fetch_refs", the_repository);
-+ if (ret) {
-+ transport_unlock_pack(transport);
-+ return ret;
-+ }
++ if (ret)
++ goto out;
}
- if (!ret)
- /*
@@ builtin/fetch.c: static int check_exist_and_connected(struct ref *ref_map)
- * time to update refs to reference the new objects.
- */
- return 0;
-- transport_unlock_pack(transport);
-- return ret;
+
+ /*
+ * Keep the new pack's ".keep" file around to allow the caller
+ * time to update refs to reference the new objects.
+ */
-+ return 0;
++ return ret;
++
++out:
+ transport_unlock_pack(transport);
+ return ret;
}
-
- /* Update local refs based on the ref values fetched from a remote */
6: 31d9f72edf ! 6: 56ecbfc9c3 fetch: merge fetching and consuming refs
@@ builtin/fetch.c: static int check_exist_and_connected(struct ref *ref_map)
/*
@@ builtin/fetch.c: static int fetch_refs(struct transport *transport, struct ref *ref_map)
- trace2_region_enter("fetch", "fetch_refs", the_repository);
- ret = transport_fetch_refs(transport, ref_map);
- trace2_region_leave("fetch", "fetch_refs", the_repository);
-- if (ret) {
-- transport_unlock_pack(transport);
-- return ret;
-- }
-+ if (ret)
-+ goto out;
+ goto out;
}
- /*
- * Keep the new pack's ".keep" file around to allow the caller
- * time to update refs to reference the new objects.
- */
-- return 0;
+- return ret;
+-
+-out:
+- transport_unlock_pack(transport);
+- return ret;
-}
-
-/* Update local refs based on the ref values fetched from a remote */
7: 84e39c847f = 7: c342fc0c69 fetch: avoid second connectivity check if we already have all objects
--
2.33.0
Attachment:
signature.asc
Description: PGP signature
