I don't really understand search in any depth, but it's interesting to observe that, in addition to the different command (SEARCH vs THREAD), those two searches are also using different search criteria ("BODY linux" vs "TEXT linux").
It might be informative to try do the SEARCH search with "TEXT linux" instead of "BODY linux", to narrow down whether the difference is due to the use of the SEARCH vs THREAD command, or the use of the "BODY" vs "TEXT" search key?
Looking at the source on master, SEARCH and THREAD both seem to be using the same search API, so at a glance it seems like they should both be using Xapian if either is. And looking at the commit dates on those functions, it doesn't look like it's changed substantially since 3.0, at least not at a level I can easily see.
I had a quick look at the RFC, and "BODY" searches just the message body, whereas "TEXT" searches both body and headers. So I wonder if the difference is that TEXT needs to open all the message files to read the headers, whereas BODY can just return results straight from the Xapian index?
I'm not sure if there's been changes to header searching (like, maybe we index more of the header content?) since 3.0, but this is getting beyond what I know off the cuff or can just casually look up.
Anyway, if you could try "UID SEARCH TEXT linux" and see if that's similarly slow to the THREAD version, that would give us a definite pointer in the right direction.
Cheers,
ellie
On Sun, Feb 23, 2020, at 10:11 PM, Frederik Himpe via Info-cyrus wrote:
I have configured Cyrus 3.0.13 with the Xapian search engine and
enabled search_fuzzy_always. This appears to work fine when I search in
the message body using the Evolution mail client, as I get a response
quickly:
<1582453709<L03163 UID SEARCH BODY linux
1582453713>* SEARCH 226927
226929 226964 226974 226999 227215 227238 [...]
L03163 OK Completed (643
msgs in 0.970 secs)
However when I search messages using the Roundcube webmail client,
Roundcube does not get a response in time and shows no results. An
strace of the imapd proceess indicates it is STATing, OPENing and
MMAPing all files in the mailbox.
This is the log:
<1582455581<A0004 UID THREAD REFS US-ASCII ALL UNSEEN TEXT Linux
1582455723>* THREAD
(229566)(229570)(229574)(229599)(229618)(229639)[...]
A0004 OK Completed (157 msgs in 11.340 secs)
So it appears Roundcube is using a different command to search. Is it
expected that this command does not use the Xapian search engine? Is
there a way to make it use it?
Some relevant snippets from imapd.conf:
sync_log: on
sync_log_channels: squatter
conversations: 1
search_engine: xapian
search_index_headers: no
search_batchsize: 8192
search_fuzzy_always: 1
defaultsearchtier: temp
tempsearchpartition-default: /var/lib/cyrus/search.temp
datasearchpartition-default: /var/lib/cyrus/search.data
cyrus.conf:
EVENTS {
squatter1 cmd="/usr/bin/nice -n 19 /usr/sbin/cyrus
squatter -z data -t temp,data" at=0517
}
DAEMON {
squatter cmd="squatter -R"
}
Regards,
--
Frederik Himpe <frederik@xxxxxxxx>
----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
----
Cyrus Home Page:
http://www.cyrusimap.org/List Archives/Info:
http://lists.andrew.cmu.edu/pipermail/info-cyrus/To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus