On Tue, Nov 27, 2018 at 5:55 PM Elijah Newren <newren@xxxxxxxxx> wrote: > > On Tue, Nov 27, 2018 at 4:16 PM Ævar Arnfjörð Bjarmason > <avarab@xxxxxxxxx> wrote: > > > > On Wed, Nov 28 2018, Bryan Turner wrote: > > > > > On Tue, Nov 27, 2018 at 3:47 PM Ævar Arnfjörð Bjarmason > > > <avarab@xxxxxxxxx> wrote: > > >> > > >> On Tue, Nov 27 2018, Bryan Turner wrote: > > >> > > >> > > > >> > Is there anything I can set, perhaps some invalid configuration > > >> > option/value, that will make "git gc" (most important) and "git > > >> > reflog" (ideal, but less important) fail when they're run in our > > >> > repositories? Hopefully at that point customers will reach out to us > > >> > for help _before_ they corrupt their repositories. > > >> > > >> $ stahp='Bryan.Turner.will.hunt.you.down.if.you.manually.run.gc' && git -c gc.pruneExpire=$stahp gc; git -c gc.reflogExpire=$stahp reflog expire > > >> error: Invalid gc.pruneexpire: 'Bryan.Turner.will.hunt.you.down.if.you.manually.run.gc' > > >> fatal: unable to parse 'gc.pruneexpire' from command-line config > > >> error: 'Bryan.Turner.will.hunt.you.down.if.you.manually.run.gc' for 'gc.reflogexpire' is not a valid timestamp > > >> fatal: unable to parse 'gc.reflogexpire' from command-line config > > > > > > Thanks for that! It looks like that does block both "git gc" and "git > > > reflog expire" without blocking "git pack-refs", "git repack" or "git > > > prune". Fantastic! The fact that it shows the invalid value means it > > > might also be possible to at least provide a useful hint that manual > > > GC is not safe. > > > > > > I appreciate your help, Ævar. > > > > No problem. I was going to add that you can set > > e.g. pack.windowMemory='some.message' to make this go for git-repack > > too, but it sounds like you don't want that. > > > > Is there a reason for why BitBucket isn't using 'git-gc'? Some other > > hosting providers use it, and if you don't run it with "expire now" or > > similarly aggressive non-default values on an active repository it won't > > corrupt anything. We did use "git gc" (sans options; its configuration was applied via "config") for several years, but there were some rough edges. The biggest one was that "git gc" has a canned set of steps it runs that can't be disabled, even when they add no value (or are actively detrimental). For us, the biggest issue was "git gc"'s insistence on trying to run "git reflog expire". That triggers locking behaviors that resulted in very frequent GC failures--and the only reflogs Bitbucket Server (by default) creates are all configured to never ex[ire or be pruned, so the effort is all wasted anyway. Counting objects: 3, done. Delta compression using up to 8 threads. Compressing objects: 100% (2/2), done. Writing objects: 100% (3/3), 270 bytes | 0 bytes/s, done. Total 3 (delta 1), reused 0 (delta 0) remote: error: cannot lock ref 'stash-refs/pull-requests/13996/merge': ref 'stash-refs/pull-requests/13996/merge' is at 2ec6a74b453d76f7a5247baa9f396361027ffdf but expected 1678f303202010c6d7e6201226df08dc8fc49ae3 remote: error: cannot lock ref 'stash-refs/pull-requests/13996/to': ref 'stash-refs/pull-requests/13996/to' is at bd32d53e4fe63b15be029085f1b6d795d526adbc but expected f55ec06e89a11b8bdbcd97680a900361307d28c4 remote: error: cannot lock ref 'stash-refs/pull-requests/14006/merge': ref 'stash-refs/pull-requests/14006/merge' is at 1cc907e2a7082033d70a164f222d3cce17a453a9 but expected ae057003d7ed7d096b5b952191d784113f25b982 remote: error: cannot lock ref 'stash-refs/pull-requests/14006/to': ref 'stash-refs/pull-requests/14006/to' is at bd32d53e4fe63b15be029085f1b6d795d526adbc but expected f55ec06e89a11b8bdbcd97680a900361307d28c4 remote: error: cannot lock ref 'stash-refs/pull-requests/14043/merge': ref 'stash-refs/pull-requests/14043/merge' is at a2e510b1b2b583b273f6d6d28e13151619e8d143 but expected 7735a5bde21815d307c68244e8fd2d67a09d5a39 remote: error: cannot lock ref 'stash-refs/pull-requests/14043/to': ref 'stash-refs/pull-requests/14043/to' is at bd32d53e4fe63b15be029085f1b6d795d526adbc but expected f55ec06e89a11b8bdbcd97680a900361307d28c4 remote: error: cannot lock ref 'stash-refs/pull-requests/14047/merge': ref 'stash-refs/pull-requests/14047/merge' is at bd4f0e9bcbed34fd9befa65763f5aee6c9ebd8ce but expected 649dea948e8e6b54506615e5d61c6779c242d5af remote: error: cannot lock ref 'stash-refs/pull-requests/14047/to': ref 'stash-refs/pull-requests/14047/to' is at bd32d53e4fe63b15be029085f1b6d795d526adbc but expected f55ec06e89a11b8bdbcd97680a900361307d28c4 remote: error: failed to run reflog To https://... f55ec06..bd32d53 master -> master (Note: That example was from when "git gc --auto" was running attached to a push, because auto-detach doesn't always detach, but our explicit "git gc" processing would fail with the same "cannot lock ref" messages.) The worktree and rerere cleanup "git gc" does is also unnecessary overhead, but was less of a concern because it wouldn't make GC _fail_. Another issue with the canned steps for "git gc" is that it means it can't be used to do specific types of cleanup on a different schedule from others. For example, we use "git pack-refs" directly to frequently pack the refs in our repositories, separate from "git repack" + "git prune" for repacking objects. That allows us to keep our refs packed better without incurring the full overhead of constantly building new packs. It may be possible to make "git gc" run more targeted operations, but even looking through the C code I couldn't find a way. > > ...assuming no other repo has this one as an alternate, which I > suspect is the issue at play. (I wrote an alternate-aware gc script > years ago when using Atlassian Stash to try to workaround this issue, > but think I only used it for a couple repos and never got around to > deploying it in prod for continuous use, probably worried I had missed > a corner case. Had meant to, but at some point the powers that be > decided to push us toward a different repository manager tool, and > I've long since forgotten most details.) "git gc" should be perfectly safe to run in a Bitbucket Server repository which has forks; we "simply" configure it to never prune (and that's applied in "config", so even if an administrator runs "git gc" manually, _as long as they don't provide explicit arguments to the contrary_, it still won't prune). When Git 2.10 or newer is installed on the server we use "git repack --keep-unreachable" to prevent writing unreachable objects as loose, and we skip running "git prune" at all. For older versions of Git, we detect where the "objects/17" heuristic would still want to run GC immediately after GC has already run and we stream the loose object IDs to "git pack-objects" to write a "garbage" pack, and then run "git prune-packed" to remove all the loose objects. It's on the backlog to actually start pruning again, even in forked repositories, but there are some other design changes we need to make first before we can ship that. (Not to mention the enormous amount of testing we need to do to ensure it's safe...) Bryan
