thoughts about conversation db and cyrus [faild to rename deleted mailbox, or how conversation db sucks again]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi Ellie and Bron,

first thank you for your ideas for the workaround and for opening the issue.

I have to apologize about the original subject, but I was a bit frustrated that
I have encountered again a problem with the conversation db.

As I have discovered jet another, not jet reported problem with conversation db (I am unsure if it's a problem with the conversation db or if only shows an other
problem some where else), I have decided to deactivate the conversation db for
the moment.

I think I should elaborate the general problem I have with this feature and the
cyrus development as i have observed it.

I am an experienced linux and cyrus administrator. I am not an software developer,
or programmer, but I understand enough about programming to fix small problems
and narrow down problems.

Cyrus code quality has grown in the last 14 years since i first set up our first
cyrus imap (2.3) server. Especially since Fastmail dedicated personal for the
cyrus project. But also automatic testing and other design decisions helped to
bring the project a big step forward.

I know we have a complex setup (murder, replication, meta- and archive partition, delayed delete, delayed expunge) so that we use combinations of features that are not
that common. So fare we are happy with our cyrus setup.

We did not encounter any data loss, and where able to fix most problems in short time. The system is very stable and expect for the slow search can't complain about the performance. So thanks again to the devs for the work and also for the community the help I received
in the last 14 years.

I like new features, but I have always to balance the advantages of these new features with the impact on stability, performance and administration overhead. In that regard I
tend to exercise caution.

While testing cyrus 3.0, on one hand neither conversation db nor sphinx or xapian search engines looked particular interesting, as I had no need to improve search speed in cyrus 2.4 with squatter ("If it isn't broken don't fix it"). On the other hand i didn't have time to rigorous test these features, so I decided not to use them as new feature
have a tendency to contain more new bugs.

After upgrading the production system to cyrus 3.0 I discovered that search was slow with squatter, conversation db did more than the documentation suggested that it does. I don't know if it was intended that conversation db was required for squatter to work or if it broke by one change to support multiple search engines but the requirement was surprising. I did try to find the commit that did break the squatter search but
failed, as was unable to compile most commits git bisect suggested.

The problems I encountered with conversation db, confirmed my initial caution not to enable conversation db without testing. But it would be unfair to blame only the implementation of
conversation db.

One problem was caused by "reconstruct -V max" not upgrading all mailboxes, which i did miss in the logs. It would be nice if ctl_conversationsdb would check mailbox version
before creating a huge conversation db file in an endless loop.

The problem with "IOERROR: conversations_audit on load:" and "IOERROR: conversations_audit on store:" is still a mystery to so it is unclear if it is really a bug in the conversation db of if it shows
an other problem with my installation.

The same is with the new problem i had no time to report jet. Deleting users i see the following error
popping up for some accounts.
"Fatal error: Internal error: assertion failed: imap/conversations.c: 2205: !status.exists"

But breaking an common use case of an other feature like "delete_mode: delayed" is an other case. This should have been fixed before it was released in the stable
cyrus version. At least a WARNING in the documentation is required.

I would like to help to improve the documentation, but there are some questions that need to be answered:

1. Which search engines and combinations are currently supported?
   Is a stand alone squatter still supported?

2. What are to pros and cons for the supported search engines?

3. Should, or to which extend should, the search engines work without conversations db? Or is enabling the conversations db a new requirement for some/all search engines?

4. Is the conversations db murder aware? And how do shared folders (one user shared one
    of his folders with other users) on the same server/cross server
    affect search results and performances

5. What is stored in the conversations db?
https://www.cyrusimap.org/dev/imap/concepts/deployment/databases.html#conversations-userid-conversations
   is incomplete as conversations db also contains hashes of mime parts.

6. Which Information is affected by conversations_expire_days

TLDR; I like cyrus, but there is some work to do regarding to conversation db and search engines,
in the field of documentation, code testing and feature interaction


--------------------------------------------------------------------------------
M.Menge                                Tel.: (49) 7071/29-70316
Universität Tübingen                   Fax.: (49) 7071/29-5912
Zentrum für Datenverarbeitung mail: michael.menge@xxxxxxxxxxxxxxxxxxxx
Wächterstraße 76
72074 Tübingen

----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus




[Index of Archives]     [Cyrus SASL]     [Squirrel Mail]     [Asterisk PBX]     [Video For Linux]     [Photo]     [Yosemite News]     [gtk]     [KDE]     [Gimp on Windows]     [Steve's Art]

  Powered by Linux