Hi Brendan On 12/12/2024 20:23, Brendan Jackman wrote:
I am developing a tool that uses Git via its CLI. The tool creates a bunch of worktrees on startup and tears them down on shutdown. When running its tests, it does this a lot. Sometimes, the git worktree add command fails with exit code 128. The output just looks like: failed to read .git/worktrees/test-worktree-xdrphX/commondir: Success Naturally, whenever I try to strace anything, the bug stops reproducing.
How annoying, I suspect it is slowing things down enough that the race condition triggering this error does not happen
[...] Any idea what's going wrong here? Is what I'm doing (i.e. creating worktrees in parallel) actually legit?
Unfortunately with the current implementation of "git worktree add" it isn't. When git tries to add a new worktree it first checks to see if it already exists. As a worktree may be on removable media it cannot simply check that the directory does not exist, it also has to check that there isn't a locked worktree at that path. To check for a locked worktree it has to list all the worktrees in the repository and if another process is creating a new worktree at the same time the process trying the list the worktrees ends up seeing an corrupt wortree because the other process hasn't finished creating all the required files yet. I think this is fixable in principle by teaching the code that lists the worktrees to ignore ".git/worktrees/*.lock" and creating new worktree under ".git/worktrees/$id.lock" and then renaming "$id.lock" to "$id" once all the files are written. There are probably some subtleties though as we would need to ensure the processes that are spawned to setup the worktree use the temporary ".lock" directory.
Anything I can do to help narrow down the issue?
If you're interested in fixing the problem the relevant code lives in worktree.c and builtin/worktree.c. The easiest solution is probably to try introduce some locking in your program so you don't run two instances of "git worktree add" at the same time.
Best Wishes Phillip
.:: Details I've built git from next on Ubuntu 24.04.1 LTS, here's the git bugreport output: https://gist.github.com/bjackman/3377add9e8d7b747d9f74b52518e9101/edit But, I also experienced this issue with the Ubuntu packaged version of Git. The application with the hacks to add logging of Git commands is here: https://github.com/bjackman/limmat/tree/git-bug Here's an example of the test failing in CI: https://github.com/bjackman/limmat/actions/runs/12303338582/job/34338222752#step:5:337 If you're willing to try and reproduce locally, in theory it should be enough to put your Git build in $PATH, clone my repo, install Cargo (https://doc.rust-lang.org/cargo/getting-started/installation.html), and then from the base of my repo run `cargo test race` in a loop, in my case it usually fails relatively quickly. Many thanks, Brendan
