John Poelstra wrote:
== CVS Branching == * 36+ hours of outage for branching* cannot do this again--must to figure out a way to do this without an outage * the second batch of cvs branches is about 50% of the way through which started 11~ hours ago * part of the delay is the pkgdb updates--a different SCM wouldn't necessarily fix the problem
Here's some inkling of the process and where we need to fix things:Mass branching happens in two parts. The first part updates the pkgdb with the new branch information. The second part updates the cvs server with the new branches from information stored in the pkgdb.
The first part had a rate of ~ 4 packages / minute.The second part can cache the pkgdb information so there's a 30s initial loading of pkgdb information followed by a rate of 3 packages / minute.
After the branching, Bill Nottingham (notting) took a look at how expensive it was to create the modules file in cvs vs the benefits it gave us. After some discussion on buildsys-list[1]_ Bill removed that functionality from our mkbranch script and made a fresh import of the module file.
Jesse subsequently made branches on the cvs server for 206 packages that hadn't branched. This took 4 minutes. Extrapolating this out, the second step will now take 2-3 hours. We could still work on making this non-blocking but it's no longer the speed bottleneck that it was before.
This leaves the first step. The only reason we have an outage during this period is to disable email sending while the branching is being done. Instead of doing this, we have two choices:
1) Add a flag to the pkgdb API to disable email sending when mass branching or other admin requests -- Could be useful for other operations as well. Could also be a security risk. 2) Add a specific API to pkgdb that mass branches packages without sending email -- likely to be faster due to being able to combine certain calls to the database.
.. _[1]: https://www.redhat.com/archives/fedora-buildsys-list/2008-April/msg00017.html
-Toshio
Attachment:
signature.asc
Description: OpenPGP digital signature
-- fedora-devel-list mailing list fedora-devel-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/fedora-devel-list
