On Wed, Sep 08 2021, Taylor Blau wrote:
> From: Ævar Arnfjörð Bjarmason <avarab@xxxxxxxxx>
>
> In preceding commits the race of renaming .idx files in place before
> .rev files and other auxiliary files was fixed in pack-write.c's
> finish_tmp_packfile(), builtin/repack.c's "struct exts", and
> builtin/index-pack.c's final(). As noted in the change to pack-write.c
> we left in place the issue of writing *.bitmap files after the *.idx,
> let's fix that issue.
>
> See 7cc8f971085 (pack-objects: implement bitmap writing, 2013-12-21)
> for commentary at the time when *.bitmap was implemented about how
> those files are written out, nothing in that commit contradicts what's
> being done here.
>
> Note that this commit and preceding ones only close any race condition
> with *.idx files being written before their auxiliary files if we're
> optimistic about our lack of fsync()-ing in this are not tripping us
> over. See the thread at [1] for a rabbit hole of various discussions
> about filesystem races in the face of doing and not doing fsync() (and
> if doing fsync(), not doing it properly).
>
> In particular, in this case of writing to ".git/objects/pack" we only
> write and fsync() the individual files, but if we wanted to guarantee
> that the metadata update was seen in that way by concurrent processes
> we'd need to fsync() on the "fd" of the containing directory. That
> concern is probably more theoretical than not, modern OS's tend to be
> more on the forgiving side than the overly pedantic side of
> implementing POSIX FS semantics.
Some weird gramma/phrasing of mine left over here, i.e. the "[...] in
this are not tripping us over.". On reflection perhaps it's better to
replace these last two paragraphs with say:
With this and preceding commits we've covered all the cases where we
wrote another auxiliary file before the *.idx file, and should thus
never have another concurrent process try to use an incomplete set
of pack-OID.* files.
This assumes that the renames we make here and elsewhere appear on
the filesystem in the order we performed them. This assumption is
known to be false in the face of pedantic POSIX FS semantics. See
the thread around [1] for discussions on that point.
We fsync() the file descriptors we're writing out for all the
auxiliary files, but we don't yet fsync() the file descriptor for
the containing directory. Therefore our data might have been written
out, but it's anyone's guess what the state of the directory
containing the file is after we write the *.idx.
In practice modern OS's are known to be forgiving on that point, so
this will probably solve races in practice for most users. It will
almost certainly make them better than they were before when we
didn't write *.idx files last. We should more generally improve our
use of fsync() to cover containing directories, but that'll
hopefully be addressed by some follow-up series.
