Re: Conversations and search

[Bron Gondwana <brong@xxxxxxxxxxxxxxxx>]

On Sat, Aug 18, 2018, at 00:08, Albert Shih wrote:

Le 16/08/2018 à 14:04:22+0200, Sebastian Hagedorn a écrit

Hi

I have a question regarding the conversations db and how it affects (Xapian)

search. In this GitHub issue

(<https://github.com/cyrusimap/cyrus-imapd/issues/2376>) I was dealing with

Xapian search always failing. The underlying reason turned out to be an

empty conversations db. When I regenerate that user's conversations db,

Xapian search works fine. So far, so good. But then I noticed this piece of

documentation:

<https://www.cyrusimap.org/imap/concepts/deployment/databases.html?highlight=conversations#conversations-userid-conversations>

Quote: "This file contains all the message-id fields from every email that

has been seen in the ***past three months***, mapping to the conversation

IDs in which this message ID has been seen, and the timestamp when it was

last seen."

This raises a number of questions. What does "has been seen" mean in this

context? And does it mean that Xapian search will always fail to find emails

that arrived more than three months ago?? That doesn't sound very useful ...

If I'm correct (I'm new with cyrus too), the conversations DB as nothing to

do with the database of xapian.

The xapian database is some file with .glass extension and contain all the

index of all your mail. Those files stand inside

  t1searchpartition-default/FIRST_LETTER_OF_LOGIN/user/LOGIN/

The conversation database is some cyrus internal database use by cyrus to

*create* the xapian index.

In other word the three months means, if you stop the xapian index now, you

have three months to restart it or you're going to need to regenerated the

xapian database from the beginning (and that take very loooong time).

Please if it's incorrect please someone correct me.

That's incorrect - the messageids are actually only used for thread calculations - so if you get a new messages more than 3 months later, it won't be threaded with related messages (conversations.db threads only, aka: JMAP and XCONV commands).

The G keys used for xapian are kept forever (at least: until the message is deleted from the index - which is not when it's expunged, but when cyr_expire cleans up the expunged record).

Bron.

--

  Bron Gondwana, CEO, FastMail Pty Ltd

  brong@xxxxxxxxxxxxxxxx

----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus