Hi,
sorry, I had sent the prior mail prematurely (hit wrong key)
and have been busy working on the resubmission...
On Thu, Apr 16, 2015 at 01:10:36PM +0200, Thomas Braun wrote:
> Am 16.04.2015 um 12:03 schrieb Andreas Mohr:
> > Hi all,
> >
> > over the years I've had the same phenomenon with various versions of msysgit
> > (now at 1.9.5.msysgit.0, on Windows 7 64bit), so I'm now sufficiently
> > confident of it being a long-standing, longer-term issue and thus I'm
> > reporting it now.
(I've had experience with this issue as early as 1.7.x, I believe).
> (CC'ing msysgit)
>
> Hi Andreas,
>
> > Since I'm doing development in a sufficiently rebase-heavy manner,
> > I seem to aggregate a lot of objects.
> > Thus, when fetching content I'm sufficiently frequently greeted with
> > a git gc run.
> > This, however, does not work fully reliably:
> >
> > Auto packing the repository for optimum performance. You may also
> > run "git gc" manually. See "git help gc" for more information.
> > Counting objects: 206527, done.
> > Delta compression using up to 4 threads.
> > Compressing objects: 100% (27430/27430), done.
> > Writing objects: 100% (206527/206527), done.
> > Total 206527 (delta 178632), reused 206527 (delta 178632)
> > Unlink of file '.git/objects/pack/pack-ab1712db0a94b5c55538d3b4cb3660cedc264c3c.pack' failed. Should I try again? (y/n) n
> > Unlink of file '.git/objects/pack/pack-ab1712db0a94b5c55538d3b4cb3660cedc264c3c.idx' failed. Should I try again? (y/n) n
> > Checking connectivity: 206527, done.
> >
> > A workable workaround for this recurring issue
> > (such a fetch will fail repeatedly,
> > thereby hampering my ability to update properly)
> > is to manually do a "git gc --auto"
> > prior to the fetch (which will then succeed).
>
> I've never had this issue. The error message from unlinking the file
> means that someone is still accessing the file and thus it can not be
> deleted (due to the implicit file locking on windows).
>
> Can you reproduce the error reliably?
It seems to be reproducible pretty reliably,
at least once git thinks it needs to repack (initiated by a fetch operation, I think),
*and* then the unlink issue successfully turned up
(which may happen perhaps every 20 fetches of a *very* rebase-heavy workflow).
Interim mail content:
I strongly suspect that git's repacking implementation
(probably unrelated to msysgit-specific deviations,
IOW, git *core* handling)
simply is buggy
in that it may keep certain file descriptors open
at least a certain time (depending on scope of implementation/objects!?)
beyond having finished its operation (rename?).
As a related note, in an unrelated application of mine
I also encountered issues on Windows
where renaming of in-use files and further use of these files/names
then failed (error code was EACCES I believe).
IOW, this seems to be an issue specific to
Windows' "special" (and sometimes quirky) filesystem handling
which probably does not turn up on many "other" platforms,
thus a historic existing implementation weakness in git's repack handling
could not be nailed down in a sufficiently easy manner.
I think I may have the order wrong, however:
Handling seems to be:
- repack needed
- counting objects
- compressing
- writing
- unlink (delete) of all prior non-repacked objects (which fails)
I have to admit that at this point in time I'm actually unsure
which higher-level operation it actually is
that gets carried out where eventually a repack *implicitly* gets triggered
(I've got a shell script here which implements clean branch updating,
where I eventually hit the problem during its daily use).
Since a standalone git gc --auto *immediately* appears to work
(after many repeated attempts of failing full-update),
this is a strong hint that (in the failure case)
it's the *PRIOR* (non-repack) operation
which has kept these objects open beyond its actual operation scope.
Suspected implementation sample code:
if (operation_needed)
{
operation_workingset set;
set.DoStuff();
if (repack_needed)
{
repack_handler repack;
repack.DoStuff();
}
}
[NOTE the very prominent scope issues in this example,
which might be the exact reason for hitting such unlink failures -
simply due to having kept file descriptors open within the working set]
I have not had a look at git source though
to actually determine whether there do exist
such severe operation scope issues
that I'm strongly contemplating.
Andreas Mohr
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html