Is "bare"ness in the context of multiple worktrees weird? Bitmap error in git gc.

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi folks,

I work with a project that often, or typically, benefits from having
multiple worktrees where users are working with different versions of
the code.

Worktrees make particular sense in this project for all the usual
reasons, plus the fact that it is a very *large* repo - multiple gigs
of repo, and very fast-moving; cloning and fetching less often is
nice.

One problem that we found users to have, early on when we introduced
worktrees, was that it was not obvious to users that there was (or why
there was) a "main" worktree, containing the actual ".git" repo, and
"subsidiary" worktrees, in the same directory location. Git by default
makes worktrees *subfolders* of the initial/main worktree/folder, but
we put them alongside each other to make other project tooling be able
to treat all worktrees consistently; in daily use there is absolutely
no difference between them, in all the "project tooling" there cannot
be a practical difference... but if you choose to delete a worktree,
and it happens to be the "special" one that contains the repo...
you've just lost stuff that you didn't expect to lose (any local
branches and stashes, basically).

Because worktree use was so useful/widespread/critical on this
project, and we already had a custom cloning process that introduced
selective refspecs etc, we introduced a special clone topology: the
initial clone is a bare repo, and that folder gets a specific clear
name (ending in .git). Then we create worktrees attached to that bare
repo.

Generally speaking, this has worked *very* well: I would recommend it
generally as a recognized/supported local-repo-setup. The most
important thing that makes this *possible* is the fact that "git
rev-parse --is-bare-repository" returns True in the bare repo folder,
where the lack of index and HEAD shouldn't bother git, and it returns
False in any one of the worktrees. It feels like things were designed
to work this way, even though I can find no explicit mention of this
topology in the docs.

However, from time to time something weird happens: Today I finally
started to understand why I was seeing a crazy GC error about bitmaps,
intermittently: It seems to be because "git gc" wants to create
bitmaps in bare repos, but can't do so properly when we're in a
partial clone... or something like that?

EG repro:
```
git clone --bare https://github.com/ksylor/ohshitgit dangit_shared.git
--filter=blob:none
git -C dangit_shared.git worktree add $PWD/dangit_wt3
cd dangit_wt3/
echo "this is some new unique blob in the repo" > new_blob.txt
git add new_blob.txt && git commit -m "new blob"
cd ../dangit_shared.git/
git gc
```

This yields, at the end of the GC run:
```
warning: Failed to write bitmap index. Packfile doesn't have full
closure (object bf86ed1b2602ac3a8d4724bcdf6707b156673aac is missing)
fatal: failed to write bitmap index
fatal: failed to run repack
```

On the other hand, running "git gc" in one of the worktrees works fine
(except you first need to delete a couple of ".tmp-*" files from the
"objects/pack" folder, if you already got the error above).

I at first thought this was a bug - but as I realized the problematic
behavior was tied to the "core.bare" setting (and its expression at
runtime through "git rev-parse --is-bare-repository"), it became more
obvious that maybe the system could/should be able to make assumptions
about the kind of repo that has this "true", and assume there are no
local-object-derived non-promisor packfiles (or whatever it is about
this example that makes things unhappy).

So, I guess I'm confused as to what "core.bare" is supposed to mean:
Is it intended to mean "there is no index nor HEAD here, and that's
good, don't worry" (in which case my setup is presumably "supported",
and the gc behavior is buggy?), or is it intended to mean "this is the
kind of repository in which there are no worktrees" (in which case I
am abusing the system and get the errors I deserve)?

CC Dscho, who made the worktree support that I rely on work about 16
years ago it seems, and Taylor Blau and Patrick Steinhardt, who I
think have made changes in the area of the code concerned with whether
or not we try to create bitmaps, and might have an opinion as to
whether the assumptions made by "gc" at the moment are unsafe, or my
use or "core.bare" in this context is wrong.

Thanks for any feedback!
Tao



[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]

  Powered by Linux