Re: xindy, texlive, and concurrency

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 





On Thu, 16 May 2019 at 01:51, Vít Ondruch <vondruch@xxxxxxxxxx> wrote:
Dear Jerry,

Although I have no idea what xindy is, I enjoyed reading your analysis.
Thx for your insightful post.



I want to say ++ to this. I have found every one of Mr James articles explaining all the things he has done to work on, debug, dead-ends, etc to be insightful and incredibly useful to track down in other software. If you have a chance, please look up some previous ones. 

Thank you again Jerry James. Your work and analysis is deeply appreciated.

 
Vít


Dne 16. 05. 19 v 3:45 Jerry James napsal(a):
> I finally found some time to look at the xindy failure to build.
> First, let me say that I've got a workaround for the problem, which
> resulted in the beautiful green colors in this xindy-enabled scratch
> build of texlive-base:
>
> https://koji.fedoraproject.org/koji/taskinfo?taskID=34877270
>
> When the build process reached the xindy part of the build, it would
> successfully build xindy itself, then go to work on some
> documentation.  This involved invoking latex on several input files.
> This marked the first (and possibly only) point in the build where
> latex was invoked, so latex.fmt had not yet been generated.  Since we
> build with %{?_smp_mflags}, several independent threads invoked latex
> at approximately the same time.  Every one of those invocations
> detected that latex.fmt was missing and tried to generate it.
>
> If you got lucky, each one of those threads would generate latex.fmt,
> then quickly consume it as it ran on its appointed input file.  If you
> didn't get lucky, one or more threads would start reading latex.fmt
> after some other thread started writing it, but before it finished
> writing it all the way.  That's why xindy would sometimes build and
> sometimes fail to build: the build process had a race condition.
>
> It is unfortunate that texlive is smart enough to detect missing
> format files and generate them, but not smart enough to stop that from
> happening multiple times concurrently.  Anyway, poor xindy has been
> maligned: none of this was xindy's fault, nor clisp's.  We, the Fedora
> packagers, threw concurrency at a job that lacked concurrency control.
> And the whole xindy_arches thing is useless: this is not an
> arch-specific problem, so removing random arches from the build
> accomplishes nothing.
>
> The workaround is to invoke latex on a dummy input file early in
> %build.  This causes latex.fmt to be generated, and then everything is
> hunky-dory when xindy is built later.
>
> My recommendation is to remove the xindy_arches conditional from the
> texlive-base and texlive spec files.  Make it unconditionallly on
> again.  Then insert something like this at the top of %build:
>
> # Make texlive generate latex.fmt, so that multiple threads do not race to
> # make it during the xindy build.
> cat > dummy.tex << EOF
> \documentclass{article}
> \begin{document}
> This is a document.
> \end{document}
> EOF
> latex dummy.tex
> rm -f dummy.*
>
> That is the substance of this pull request:
>
> https://src.fedoraproject.org/rpms/texlive-base/pull-request/6
>
> Also, I should be able to push a clisp build with s390x support to F30
> stable tomorrow.  I, personally, would really appreciate having xindy
> reappear in F30, due to Sphinx assuming it is available.
>
> Regards,
_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx


--
Stephen J Smoogen.

_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Index of Archives]     [Fedora Announce]     [Fedora Users]     [Fedora Kernel]     [Fedora Testing]     [Fedora Formulas]     [Fedora PHP Devel]     [Kernel Development]     [Fedora Legacy]     [Fedora Maintainers]     [Fedora Desktop]     [PAM]     [Red Hat Development]     [Gimp]     [Yosemite News]

  Powered by Linux