On Thu, May 16, 2019 at 09:11:21AM +0200, Dridi Boukelmoune wrote: > On Thu, May 16, 2019 at 3:46 AM Jerry James <loganjerry@xxxxxxxxx> wrote: > > > > I finally found some time to look at the xindy failure to build. > > First, let me say that I've got a workaround for the problem, which > > resulted in the beautiful green colors in this xindy-enabled scratch > > build of texlive-base: > > > > https://koji.fedoraproject.org/koji/taskinfo?taskID=34877270 > > > > When the build process reached the xindy part of the build, it would > > successfully build xindy itself, then go to work on some > > documentation. This involved invoking latex on several input files. > > This marked the first (and possibly only) point in the build where > > latex was invoked, so latex.fmt had not yet been generated. Since we > > build with %{?_smp_mflags}, several independent threads invoked latex > > at approximately the same time. Every one of those invocations > > detected that latex.fmt was missing and tried to generate it. > > > > If you got lucky, each one of those threads would generate latex.fmt, > > then quickly consume it as it ran on its appointed input file. If you > > didn't get lucky, one or more threads would start reading latex.fmt > > after some other thread started writing it, but before it finished > > writing it all the way. That's why xindy would sometimes build and > > sometimes fail to build: the build process had a race condition. > > So this is a build system bug from upstream. If concurrency introduces > a race condition then source-target dependencies are lacking in the > build system. Depending on the build system it may not be upstream's > fault, but the tooling itself. > > A workaround for the concurrency problem would be an atomic write of > the generated file. This way even when it is generated multiple times > while others try to read it they either see a complete or missing file. > > The only way I'm aware of for this workaround would be to write to a > temp file on the same filesystem, and then use mv to rename it > atomically. Yes! Check-if-exists, write-to-tmpfile, atomic-rename. If the rename fails, someone else won the race, so discard the tmpfile, continue as usual. So this isn't really a bug in the build system, but a bug in latex which bungles creation of what is essentially a cache file. (One expects that a tool like latex may be invoked more than once at the same time and anything it does internally is its own business). Let me also add my +1 to the write-up itself, a very nice story. Zbyszek > > It is unfortunate that texlive is smart enough to detect missing > > format files and generate them, but not smart enough to stop that from > > happening multiple times concurrently. Anyway, poor xindy has been > > maligned: none of this was xindy's fault, nor clisp's. We, the Fedora > > packagers, threw concurrency at a job that lacked concurrency control. > > And the whole xindy_arches thing is useless: this is not an > > arch-specific problem, so removing random arches from the build > > accomplishes nothing. > > > > The workaround is to invoke latex on a dummy input file early in > > %build. This causes latex.fmt to be generated, and then everything is > > hunky-dory when xindy is built later. > > > > My recommendation is to remove the xindy_arches conditional from the > > texlive-base and texlive spec files. Make it unconditionallly on > > again. Then insert something like this at the top of %build: > > > > # Make texlive generate latex.fmt, so that multiple threads do not race to > > # make it during the xindy build. > > cat > dummy.tex << EOF > > \documentclass{article} > > \begin{document} > > This is a document. > > \end{document} > > EOF > > latex dummy.tex > > rm -f dummy.* > > > > That is the substance of this pull request: > > > > https://src.fedoraproject.org/rpms/texlive-base/pull-request/6 > > > > Also, I should be able to push a clisp build with s390x support to F30 > > stable tomorrow. I, personally, would really appreciate having xindy > > reappear in F30, due to Sphinx assuming it is available. > > > > Regards, > > -- > > Jerry James > > http://www.jamezone.org/ > > _______________________________________________ > > devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx > > To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx > > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html > > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > > List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx > _______________________________________________ > devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx > To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
