On Mon, Oct 31 2022, Mark Hills wrote: > On Mon, 31 Oct 2022, Ævar Arnfjörð Bjarmason wrote: > >> >> On Mon, Oct 31 2022, Mark Hills wrote: >> >> > Our use case: we commit some compiled objects to the repo, where compiling >> > is either slow or requires software which is not always available. >> > >> > Since upgrading Git 2.26.3 -> 2.32.4 (as part of Alpine Linux OS upgrade) >> > we are noticing a change in build behaviour. >> > >> > Now, after a "git clone" we find the Makefile intermittently attempting >> > (and failing) some builds that are not intended. >> > >> > Indeed, Make is acting reasonably as the source file is sometimes >> > marginally newer than the destination (both checked out by Git), example >> > below. >> > >> > I've never had to consider consistency timestamps within a Git checkout >> > until now. >> > >> > It's entirely possible there's _never_ a guarantee of consistency here. >> > >> > But then something has certainly changed in practice, as this fault has >> > gone from never happening to now every couple of days. >> > >> > Imaginging I can't be the first person to encounter this, I searched for >> > existing threads or docs, but overwhemingly the results were question of >> > Git tracking the timestamps (as part of the commit) which this is not; >> > it's consistency within one checkout. >> > >> > $ git clone --depth 1 file:///path/to/repo.git >> > >> > $ stat winner.jpeg >> > File: winner.jpeg >> > Size: 258243 Blocks: 520 IO Block: 4096 regular file >> > Device: fd07h/64775d Inode: 33696 Links: 1 >> > Access: (0644/-rw-r--r--) Uid: ( 106/ luthier) Gid: ( 106/ luthier) >> > Access: 2022-10-31 16:05:17.756858496 +0000 >> > Modify: 2022-10-31 16:05:17.756858496 +0000 >> > Change: 2022-10-31 16:05:17.756858496 +0000 >> > Birth: - >> > >> > $ stat winner.svg >> > File: winner.svg >> > Size: 52685 Blocks: 112 IO Block: 4096 regular file >> > Device: fd07h/64775d Inode: 33697 Links: 1 >> > Access: (0644/-rw-r--r--) Uid: ( 106/ luthier) Gid: ( 106/ luthier) >> > Access: 2022-10-31 16:05:17.766859030 +0000 >> > Modify: 2022-10-31 16:05:17.766859030 +0000 >> > Change: 2022-10-31 16:05:17.766859030 +0000 >> > Birth: - >> > >> > Elsewhere in the repository, it's clear the timestamps are not consistent: >> > >> > $ stat Makefile >> > File: Makefile >> > Size: 8369 Blocks: 24 IO Block: 4096 regular file >> > Device: fd07h/64775d Inode: 33655 Links: 1 >> > Access: (0644/-rw-r--r--) Uid: ( 106/ luthier) Gid: ( 106/ luthier) >> > Access: 2022-10-31 16:05:51.628660212 +0000 >> > Modify: 2022-10-31 16:05:17.746857963 +0000 >> > Change: 2022-10-31 16:05:17.746857963 +0000 >> > Birth: - >> >> I think you're almost certainly running into the parallel checkout, >> which is new in that revision range. Try tweaking checkout.workers and >> checkout.thresholdForParallelism (see "man git-config"). > > Thanks, it will be interesting to try this and I'll report back. FWIW I was under the impression that we'd made it the default, so unless you opted-in it's probably not that. >> I can't say without looking at the code/Makefile (and even then, I don't >> have time to dig here:), but if I had to bet I'd say that your >> dependencies have probably always been broken with these checked-in >> files, but they happend to work out if they were checked out in sorted >> order. >> >> And now with the parallel checkout they're not guaranteed to do that, as >> some workers will "race ahead" and finish in an unpredictable order. > > These are very simple Makefile rules, I don't think these dependencies are > broken; but your theory is in good alignment with the observed behaviour. > > For example, the rule from the recent case above is: > > %.jpeg: %.png > convert $< $(IMFLAGS) $@ > > %.png: %.svg > inkscape --export-type=png --export-filename=$@ $< Grom a glance those don't seem broken to me, but I don't know how it interacts with your built assets. So e.g. if you are checking in your *.jpeg files those will be more recent than either the *.png or source *.svn, so they won't be built. This is fast getting out of scope of Git-specific advice, but you should run "make --debug" (there's also sub-debug flags) to see if make's idea of the dependency graph matches yours.
