The commit-graph feature is not useful to end users until the commit-graph file is maintained automatically by Git during normal upkeep operations. One natural place to trigger this write is during 'git gc'. Before automatically generating a commit-graph, we need to be able to verify the contents of a commit-graph file. Integrate commit-graph checks into 'fsck' that check the commit-graph contents against commits in the object database. Thanks, Jakub, for the feedback on the RFC. I think there are still some things to decide at a high-level before we dig too far into the review. Specifically, the integration points in 'fsck', 'gc', and 'fetch' are worth considering our alternatives. For 'fsck', the current integration is minimal: if core.commitGraph is true, then 'git fsck' calls 'git commit-graph verify --object-dir=X' for the objects directory and every alternate. There are a few options to consider here: 1. Keep this behavior: we should always check the commit-graph if it exists. 2. Add a --[no-]commit-graph argument to 'fsck' that toggles the commit-graph verification. 3. Remove all direct integration between 'fsck' and 'commit-graph' and instead rely on users checking 'git commit-graph verify' manually when they suspect a problem with the commit-graph file. While this option is worth considering, it is my least favorite since it requires more from users. For 'gc' and 'fetch', these seemed like natural places to update the commit-graph file. Relative to the other maintenance that occurs during these commands, the 'git commit-graph write' command is fast, especially for incremental updates; only the "new" commits are walked when computing generation numbers and other metadata for the commit-graph file. The behavior in this patch series does the following: 1. Near the end of 'git gc', run 'git commit-graph write'. The location of this code assumes that a 'git gc --auto' has not terminated early due to not meeting the auto threshold. 2. At the end of 'git fetch', run 'git commit-graph write'. This means that every reachable commit will be in the commit-graph after a a successful fetch, which seems a reasonable frequency. Then, the only times we would be missing a reachable commit is after creating one locally. There is a problem with the current patch, though: every 'git fetch' call runs 'git commit-graph write', even if there were no ref updates or objects downloaded. Is there a simple way to detect if the fetch was non-trivial? One obvious problem with this approach: if we compute this during 'gc' AND 'fetch', there will be times where a 'fetch' calls 'gc' and triggers two commit-graph writes. If I were to abandon one of these patches, it would be the 'fetch' integration. A 'git gc' really wants to delete all references to unreachable commits, and without updating the commit-graph we may still have commit data in the commit-graph file that is not in the object database. In fact, deleting commits from the object database but not from the commit-graph will cause 'git commit-graph verify' to fail! I welcome discussion on these ideas, as we are venturing out of the "pure data structure" world and into the "user experience" world. I am less confident in my skills in this world, but the feature is worthless if it does not improve the user experience. Thanks, -Stolee Derrick Stolee (12): Commits 01-07 focus on the 'git commit-graph verify' subcommand. These are ready for full, rigorous review. commit-graph: add 'verify' subcommand commit-graph: verify file header information commit-graph: parse commit from chosen graph commit-graph: verify fanout and lookup table commit: force commit to parse from object database commit-graph: load a root tree from specific graph commit-graph: verify commit contents against odb Commit 08 integrates 'git commit-graph verify' into fsck. The work here is the minimum integration possible. (See above discussion of options.) fsck: verify commit-graph Commit 09 introduces a new '--reachable' option only to make the calls from 'gc' and 'fetch' simpler. Commits 10-11 integrate writing the commit-graph into 'gc' and 'fetch', respectively. (See above disucssion.) commit-graph: add '--reachable' option gc: automatically write commit-graph files fetch: compute commit-graph by default Commit 12 simply deletes sections from the "Future Work" section of the commit-graph design document. commit-graph: update design document Documentation/config.txt | 10 ++ Documentation/git-commit-graph.txt | 14 ++- Documentation/git-fsck.txt | 3 + Documentation/git-gc.txt | 4 + Documentation/technical/commit-graph.txt | 22 ---- builtin/commit-graph.c | 79 +++++++++++++- builtin/fetch.c | 13 +++ builtin/fsck.c | 21 ++++ builtin/gc.c | 8 ++ commit-graph.c | 175 ++++++++++++++++++++++++++++++- commit-graph.h | 2 + commit.c | 13 ++- commit.h | 1 + t/t5318-commit-graph.sh | 25 +++++ 14 files changed, 353 insertions(+), 37 deletions(-) base-commit: 34fdd433396ee0e3ef4de02eb2189f8226eafe4e -- 2.16.2.329.gfb62395de6
