Ok, I'll stick to Xapian and make more tests, thanx
Sonicle S.r.l. : http://www.sonicle.com
eXoplanets : https://gabrielebulfon.bandcamp.com/album/exoplanets
Da: Robert Stepanek <rsto@xxxxxxxxxxxxxxxx>
A: Gabriele Bulfon via Info <info@xxxxxxxxxxxxxxxxxx>
Data: 11 gennaio 2021 21.52.00 CET
Oggetto: Re: Moving to Xapian
On Mon, Jan 11, 2021, at 6:36 PM, Gabriele Bulfon via Info wrote:Ok, what I mean: do you think Xapian (once I enable the search_fuzzy_always) is slower than non-xapian as in 2.5.11?I don't know, I never worked with v2.5. I would be surprised if Xapian-backed search is slower, though. With the multi-tiered approach Xapian search, you can make it plenty fast by storing recently indexed mail on a memory filesystem, then SSD for your archives.What is the correct solution for full-text search on 3.2.5?The only reasonably fast search backend for 3.2.5 is Xapian. 3.4 should reintroduce the option to also use the squat backend.GabrieleSonicle S.r.l. : http://www.sonicle.comeXoplanets : https://gabrielebulfon.bandcamp.com/album/exoplanetsDa: Robert Stepanek <rsto@xxxxxxxxxxxxxxxx>
A: Gabriele Bulfon via Info <info@xxxxxxxxxxxxxxxxxx>
Data: 11 gennaio 2021 16.58.21 CET
Oggetto: Re: Moving to XapianThe issue with the squat backend in version 3 is that it actually is not called at all. Non-Xapian search in 3.0 to 3.3 degrades to a slow linear search over all messages of a mailbox. The pull request I mentioned earlier in this thread aims to fix that for 3.4.I am not aware of other changes to the squat backend, so I can't tell about any regressions in incremental indexing. Once we fixed the squat backend for 3.4, it should be simple to determine if there is an issue with incremental indexing.Cheers,RobertOn Mon, Jan 11, 2021, at 4:51 PM, Gabriele Bulfon via Info wrote:That was it! Thanks!Tough I notice it's slower than 2.5.11+squatter.It's actually slower on normal search/sort without body search.I was suggested to move to 3.x+xapian because squatter indexes are buggy, and infact on 2.5.11 I have lots of squatter crash during incremental indexing, so I had to create a specific batch to index every folder singularly and skip any folder where squatter would crash.Now I find xapian is slower? Is it true? Is there anything I can do to make it faster?Or is sqatter indexing no more buggy on 3.2.5?GabrieleSonicle S.r.l. : http://www.sonicle.comeXoplanets : https://gabrielebulfon.bandcamp.com/album/exoplanetsDa: Robert Stepanek <rsto@xxxxxxxxxxxxxxxx>
A: Gabriele Bulfon via Info <info@xxxxxxxxxxxxxxxxxx>
Data: 11 gennaio 2021 9.25.06 CET
Oggetto: Re: Moving to XapianOn Mon, Jan 11, 2021, at 9:02 AM, Sebastian Hagedorn wrote:Have you enabled search_fuzzy_always? If not, do so and repeat your test.
Other than changing your client to use FUZZY for SEARCH, this is the right approach.There are several issues with Xapian currently, the big one being that by default it is only used for fuzzy searches.Just curious, what issues are you referring to?On a related note: there is a fix for squat backend search in the works and it looks quite promising. I'll backport it to the 3.4 branch https://github.com/cyrusimap/cyrus-imapd/pull/3316
