On Mon, 2020-02-10 at 09:46 -0500, Derrick Stolee wrote: > Part of my initial plan was to have this incremental file format. > The commit-graph uses a very similar mechanism. The difference may > be that you likely allow multiple .midx files found by scanning the > pack directory, Right, just scan and use any midx that exist, then compare the packs in there against all the packs found, and then remove any packs that actually *are* in an midx from the search list. That leaves you with all information, but optimised by midx where possible. > but I would expect something like the > "commit-graph-chain" file that provides an ordered list of the > incremental files. This can be important for deciding when to merge > layers or delete old files, and would be critical to the possibility > of converting reachability bitmaps to rely on a stable object order > stored in the multi-pack-index instead of pack-order. Right, if we delete then we have to also remove any midx covering the deleted pack, that's pretty rare in bup as a backup tool though. > The reason the multi-pack-index has not become incremental is that > VFS for Git no longer needs to write it very often. We write the > entire multi-pack-index during a background job that triggers once > per day. If we needed to write it more frequently, then the incremental > format would be more important to us. So, wait, what if a new pack is created? Does it just get used in addition to the multi-pack-index, if it's not covered by it, like I described above? If so, I guess it wouldn't actually really matter here. I was afraid (but didn't check yet) that git would always use only the single multi- pack-index file, and not also search additional packs, so that it always has to be maintained in "perfect order" ... > That said: if someone wanted to contribute an incremental format, > then I would be happy to review it! I might still get motivated to do so :-) johannes
