Thanks Michael - I don't have much time to give a longer response right now, but this is really valuable feedback into what we focus on over the next few months and into the 3.2 release plans. I really appreciate you taking the time to spell this out in detail.
Cheers,
Bron.
On Tue, Mar 5, 2019, at 02:17, Michael Menge wrote:
Hi Ellie and Bron,first thank you for your ideas for the workaround and for opening the issue.I have to apologize about the original subject, but I was a bitfrustrated thatI have encountered again a problem with the conversation db.As I have discovered jet another, not jet reported problem withconversation db(I am unsure if it's a problem with the conversation db or if onlyshows an otherproblem some where else), I have decided to deactivate the conversation db forthe moment.I think I should elaborate the general problem I have with thisfeature and thecyrus development as i have observed it.I am an experienced linux and cyrus administrator. I am not ansoftware developer,or programmer, but I understand enough about programming to fix small problemsand narrow down problems.Cyrus code quality has grown in the last 14 years since i first set upour firstcyrus imap (2.3) server. Especially since Fastmail dedicated personal for thecyrus project. But also automatic testing and other design decisions helped tobring the project a big step forward.I know we have a complex setup (murder, replication, meta- and archivepartition,delayed delete, delayed expunge) so that we use combinations offeatures that are notthat common. So fare we are happy with our cyrus setup.We did not encounter any data loss, and where able to fix mostproblems in short time.The system is very stable and expect for the slow search can'tcomplain about the performance.So thanks again to the devs for the work and also for the communitythe help I receivedin the last 14 years.I like new features, but I have always to balance the advantages ofthese new featureswith the impact on stability, performance and administration overhead.In that regard Itend to exercise caution.While testing cyrus 3.0, on one hand neither conversation db norsphinx or xapian searchengines looked particular interesting, as I had no need to improvesearch speed incyrus 2.4 with squatter ("If it isn't broken don't fix it"). On theother hand i didn'thave time to rigorous test these features, so I decided not to usethem as new featurehave a tendency to contain more new bugs.After upgrading the production system to cyrus 3.0 I discovered thatsearch was slow withsquatter, conversation db did more than the documentation suggestedthat it does.I don't know if it was intended that conversation db was required forsquatter to workor if it broke by one change to support multiple search engines butthe requirementwas surprising. I did try to find the commit that did break thesquatter search butfailed, as was unable to compile most commits git bisect suggested.The problems I encountered with conversation db, confirmed my initialcaution not to enableconversation db without testing. But it would be unfair to blame onlythe implementation ofconversation db.One problem was caused by "reconstruct -V max" not upgrading allmailboxes, which i didmiss in the logs. It would be nice if ctl_conversationsdb would checkmailbox versionbefore creating a huge conversation db file in an endless loop.The problem with "IOERROR: conversations_audit on load:" and "IOERROR:conversations_audit on store:"is still a mystery to so it is unclear if it is really a bug in theconversation db of if it showsan other problem with my installation.The same is with the new problem i had no time to report jet. Deletingusers i see the following errorpopping up for some accounts."Fatal error: Internal error: assertion failed: imap/conversations.c:2205: !status.exists"But breaking an common use case of an other feature like "delete_mode:delayed" isan other case. This should have been fixed before it was released inthe stablecyrus version. At least a WARNING in the documentation is required.I would like to help to improve the documentation, but there are somequestions that need to be answered:1. Which search engines and combinations are currently supported?Is a stand alone squatter still supported?2. What are to pros and cons for the supported search engines?3. Should, or to which extend should, the search engines work withoutconversations db?Or is enabling the conversations db a new requirement for some/allsearch engines?4. Is the conversations db murder aware? And how do shared folders(one user shared oneof his folders with other users) on the same server/cross serveraffect search results and performances5. What is stored in the conversations db?https://www.cyrusimap.org/dev/imap/concepts/deployment/databases.html#conversations-userid-conversationsis incomplete as conversations db also contains hashes of mime parts.6. Which Information is affected by conversations_expire_daysTLDR; I like cyrus, but there is some work to do regarding toconversation db and search engines,in the field of documentation, code testing and feature interaction--------------------------------------------------------------------------------M.Menge Tel.: (49) 7071/29-70316Universität Tübingen Fax.: (49) 7071/29-5912Zentrum für Datenverarbeitung mail:michael.menge@xxxxxxxxxxxxxxxxxxxxWächterstraße 7672074 Tübingen----Cyrus Home Page: http://www.cyrusimap.org/List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/To Unsubscribe:https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
--
Bron Gondwana, CEO, FastMail Pty Ltd
brong@xxxxxxxxxxxxxxxx
---- Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
