On Fri, 09 Jun 2006 07:37:21 -0400, seth vidal wrote: > On Fri, 2006-06-09 at 12:31 +0100, David Woodhouse wrote: > > On Thu, 2006-06-08 at 08:11 -0400, Dan Williams wrote: > > > I think we should bump up the timeout here, actually. It's not a > > > problem on the builders, but an issue with the server and Extras > > > release scripts. > > > > Can't we just fix the locking? Surely we don't need to lock the repo for > > the _whole_ period of the upload. You can upload the new packages, then > > lock, then create metadata, then unlock. Danger. It would be possible for the push script to fetch partial uploads from the needsign repo. Unless you upload to a tmp dir, then lock, move, createrepo, unlock. > > You don't need it locked for > > more than the few seconds it takes to update the metadata for the new > > packages, surely? > > it takes longer than a few seconds to update the metadata. No repo is locked when repodata and repoview are updated by the push script. The push script locks the needsign queue only for a short period. See my recent message to buildsys-list, which explains it. There is only one lock file per dist repo, which blocks plague from updating a needsign repository while the push script is accessing it and vice versa. This file is locked only for the short time it takes to enter the passphrase, sign packages and let them install into the local master repository. The time-consuming operations (repomanage, createrepo, repoview, sync to public master repo) are performed on the local master repo, where no locking is implemented. Btw, it would not be possible to protect any yum/mock clients from accessing the public (!) master repository while a sync is in progress. You could only play tricks like doing multiple syncs, first a no-delete sync of all new packages, excluding repodata, then another sync with the new repodata, trying to make the files available as quickly as possible, then another sync to delete gone files. And it still would not be bullet-proof -- adding files plus updating metadata would need to be an atomic operation. One major problem with the push script was that _it modified rpms_ in the needsign repo. It signed them, thereby invalidating the metadata. It _moved_ them away into the local master repo, leaving build servers with a temporarily broken needsign repo until the updates arrived at download.fedora.redhat.com. (The older push script even mailed the build report long before the actual sync.) However, the situation has changed last night, and I've modified the new push script even further to work on copies of rpms. Packages in the needsign repo now are not modified or moved by the push script. Pushed and unneeded builds expire after some time (48 hours currently). -- fedora-extras-list mailing list fedora-extras-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/fedora-extras-list
