xindy, texlive, and concurrency

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



I finally found some time to look at the xindy failure to build.
First, let me say that I've got a workaround for the problem, which
resulted in the beautiful green colors in this xindy-enabled scratch
build of texlive-base:

https://koji.fedoraproject.org/koji/taskinfo?taskID=34877270

When the build process reached the xindy part of the build, it would
successfully build xindy itself, then go to work on some
documentation.  This involved invoking latex on several input files.
This marked the first (and possibly only) point in the build where
latex was invoked, so latex.fmt had not yet been generated.  Since we
build with %{?_smp_mflags}, several independent threads invoked latex
at approximately the same time.  Every one of those invocations
detected that latex.fmt was missing and tried to generate it.

If you got lucky, each one of those threads would generate latex.fmt,
then quickly consume it as it ran on its appointed input file.  If you
didn't get lucky, one or more threads would start reading latex.fmt
after some other thread started writing it, but before it finished
writing it all the way.  That's why xindy would sometimes build and
sometimes fail to build: the build process had a race condition.

It is unfortunate that texlive is smart enough to detect missing
format files and generate them, but not smart enough to stop that from
happening multiple times concurrently.  Anyway, poor xindy has been
maligned: none of this was xindy's fault, nor clisp's.  We, the Fedora
packagers, threw concurrency at a job that lacked concurrency control.
And the whole xindy_arches thing is useless: this is not an
arch-specific problem, so removing random arches from the build
accomplishes nothing.

The workaround is to invoke latex on a dummy input file early in
%build.  This causes latex.fmt to be generated, and then everything is
hunky-dory when xindy is built later.

My recommendation is to remove the xindy_arches conditional from the
texlive-base and texlive spec files.  Make it unconditionallly on
again.  Then insert something like this at the top of %build:

# Make texlive generate latex.fmt, so that multiple threads do not race to
# make it during the xindy build.
cat > dummy.tex << EOF
\documentclass{article}
\begin{document}
This is a document.
\end{document}
EOF
latex dummy.tex
rm -f dummy.*

That is the substance of this pull request:

https://src.fedoraproject.org/rpms/texlive-base/pull-request/6

Also, I should be able to push a clisp build with s390x support to F30
stable tomorrow.  I, personally, would really appreciate having xindy
reappear in F30, due to Sphinx assuming it is available.

Regards,
-- 
Jerry James
http://www.jamezone.org/
_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx




[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Index of Archives]     [Fedora Announce]     [Fedora Users]     [Fedora Kernel]     [Fedora Testing]     [Fedora Formulas]     [Fedora PHP Devel]     [Kernel Development]     [Fedora Legacy]     [Fedora Maintainers]     [Fedora Desktop]     [PAM]     [Red Hat Development]     [Gimp]     [Yosemite News]

  Powered by Linux