Another 2.4 upgrade horror story

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi,

about three weeks ago we upgraded our Cyrus installation from 2.3.x to 2.4.16. We were aware of the reindexing issue, so we took precautionary measures, but they didn't help a lot. We've got about 7 TB of mail data for almost 200,000 mailboxes. We did the upgrade on a Sunday and had told our users that mail access wouldn't be possible for the whole day. After the actual software upgrade we ran distributed scripts that triggered the index upgrades. We started with the largest mailboxes. The idea was that after those that took the longest had been upgraded, the rest should be OK overnight and early Monday. However, even though our storage infrastructure was kept at 99 % I/O saturation, progress was much slower than anticipated.

Ultimately the server was virtually unuseable for the whole Monday and parts of Tuesday. The last mailbox was finally upgraded on Thursday, although on Wednesday most things were already working normally.

I realize that some of our problems were caused by infrastructure that's not up to current standards, but nonetheless I would really urge you to never again use an upgrade mechanism like that. Give admins a chance to upgrade indexes in the background and over time.

Sebastian
--
    .:.Sebastian Hagedorn - RZKR-W (Gebäude 133), Zimmer 2.02.:.
                .:.Regionales Rechenzentrum (RRZK).:.
.:.Universität zu Köln / Cologne University - ✆ +49-221-470-89578.:.

Attachment: p7sReT7xO6RrQ.p7s
Description: S/MIME cryptographic signature

----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

[Index of Archives]     [Cyrus SASL]     [Squirrel Mail]     [Asterisk PBX]     [Video For Linux]     [Photo]     [Yosemite News]     [gtk]     [KDE]     [Gimp on Windows]     [Steve's Art]

  Powered by Linux