Le 25/09/2024 à 17:39:36-0600, Nels Lindquist a écrit Hi, > On 2024-09-25 1:23 PM, Albert Shih wrote: > > Hi everyone. > > > > So I would like to know what would be the most efficient method to > > bootstrap squatter. > > > > On my new server after sync almost all the mail I got, I try to «bootstrap» > > squatter, meaning I launch the command squatter. But it's take lot of time. > > So I try to run 64 threads (the server got 32 cores) of squatter, but still > > after 10 h I just run 10% of the mailbox. > > > > Is they are any way to speed up ? Or the only solution is to wait ? > > > > Regards > > I presume you have conversations.db enabled? It's required for xapian to > work. Use twoskip for the conversations db format. Yes. > > You could try rebuilding the conversations databases with > "ctl_conversationsdb -v -b -r". > > Did you try increasing search_batchsize? That was the "one thing" that fixed > it for me: Indeed !!!. I'v put 524288 for the search_batchsize and I think I will be over it in few hours. Big thanks. > > On 2024-04-24 2:41 AM, a.lobanov@xxxxxxxxxxxx wrote: > > > BTW. I see a strange issue in Xapian initial indexing. We have many > > folders with 10,000 and more messages. Say, N messages. If I set > > "search_batchsize" to value M less than N to prevent long mailbox > > locking in live server, squatter re-enters the folder many times > > more than N/M, with permanent intensive writing to search partition. > > So, I had to set search_batchsize to 1,000,000. How do you launch squatter ? Because I intend to launch squatter using sync_log so as I understand (but maybe I'm wrong that's why I ask), squatter are just going to use what inside the log file and index only the new mail. > Run "squatter -v -i" in a screen/tmux window so you can see what it's doing; > if you observe repeated processing of the same folder/mailbox this could be > your answer. And if it's the case what should I do ? Regards -- Albert SHIH 🦫 🐸 Observatoire de Paris France Heure locale/Local time: jeu. 26 sept. 2024 11:43:17 CEST ------------------------------------------ Cyrus: Info Permalink: https://cyrus.topicbox.com/groups/info/Tb146f45ff23d5ffa-Mdbc49bf2606c26fb475e3a4d Delivery options: https://cyrus.topicbox.com/groups/info/subscription
