On Wed, 23 Feb 2022, Maarten de Vries via arch-general wrote:
On Wed, 23 Feb 2022 at 11:40, Knut Ahlers via arch-general <
arch-general@xxxxxxxxxxxxxxxxxxx> wrote:
Hey there,
Through the latest update in glibc an issue / question emerged for me
and maybe someone on this list has some input for me how to solve
this:
We do have a repository with custom built packages (for example a
custom version of ffmpeg we need for internal purposes)… These
packages obviously do have dependencies on libraries and break when
there is a change in a library. In this case: "version `GLIBC_2.34'
not found".
Is there anyone who solved this already? How is it solved in the
Archlinux core / community repo?
What is your trigger to rebuild packages and prevent running into some
command not finding its dependencies? So how do you know when to
rebuild [some binary package depending on glibc] without manually
keeping track of the glibc version you've built the package against
and then triggering the re-build when any of the underlying library
changes its version?
How does the Archlinux community/core/extra build system keep track of
this and knows when to rebuild ffmpeg because glibc changed?
Thanks in advance for any input!
Cheers,
Knut
--
Knut Ahlers
Software & Infrastructure Developer
Web & Blog: https://ahlers.me/
GPG-Key: 0xD215AD7D
Hey,
I'm not related to the Arch Linux project, but I manage a few custom
repositories with additional packages for internal use. I've solved the
problem you describe by mirroring the Arch Linux repositories, and building
the custom packages against that mirror.
Then, about once a week, I run a script which updates the mirror in a
staging location. The script checks the .BUILDINFO for all custom packages.
That gives the version number of dependencies that were installed when the
custom package was built. Those versions are checked against the versions
in the synced Arch Linux repositories. If any of the runtime dependencies
is updated, the package and all its reverse dependencies are flagged to be
rebuild. Then, all flagged packages are rebuild in dependency order in a
clean container (which is configured to use the staging repository).
When the build is done, the staging repository becomes the live mirror.
This is done by overwriting a symlink to make it atomic. I also use btrfs
snapshots to create a staging repository to avoid copying it every time for
a rebuild.
Note that this approach rebuilds more often than necessary. You could
further inspect library dependencies of ELF files if you want to reduce the
amount of rebuilds. However, I prefer to err on the safe side in case ABI
breakage happened without matching soname bump, at the cost of doing more
rebuilds than necessary.
Maybe you can take some inspiration from this :)
Kind regards,
Maarten de Vries
Hi Knut,
I have a similar problem, which I solved a little different: I put as many
versions as possible on my dependencies (i.e. depend on "foo=1.2.3"
instead of just "foo") and use "libfoo.so=..." dependencies instead of
"foo=..." wherever possible. This has the same disadvantage, that you may
recompile more often than necessary, but it safes you from already
installing broken packages (i.e. at latest, you notice during "pacman
-Syu", that you should recompile - and not when you try to run your
software). To automate the recompile, I use a similar approach as Maarten:
I scrape the sync database of pacman, extract the provided versions and
update them at the PKGBUILDs in my repo as necessary and recompile the
packages. This runs much more often than weekly on my machine, though (I
think, I set the timer to once an hour or so).
regards,
Erich
