Hi,
Unfortunately not, but I cannot locate the specific emails, which cause this.
I cannot provide too much useful information.
It seems, when I put a lot of v in the options of squatter, it can squatter all the messages, at least there is no assertion and it did not stop.
# incremental index changed mailboxes (fulltext) approximately every hour
squatter1 cmd="/usr/sbin/cyrus squatter -vvvvv -p -i" period=180
# reindex all mailboxes (fulltext) daily
squattera cmd="/usr/sbin/cyrus squatter -vvvvv -p" at=0130
In the other hand, the log contains additional info, around the problematic folders.
Like this:
2022-11-13T01:32:11.937642+01:00 cyrus36 cyrus/squatter[19126]: indexing mailbox user/csaba/Sent...
2022-11-13T01:32:12.071267+01:00 cyrus36 cyrus/squatter[19126]: squat: opening document part ''
2022-11-13T01:32:12.071595+01:00 cyrus36 cyrus/squatter[19126]: squat: writing 10 bytes into message 32091
2022-11-13T01:32:12.078024+01:00 cyrus36 cyrus/squatter[19126]: squat: opening document part 'h32091'
2022-11-13T01:32:12.078185+01:00 cyrus36 cyrus/squatter[19126]: squat: writing 27 bytes into message 32091
2022-11-13T01:32:12.083676+01:00 cyrus36 cyrus/squatter[19126]: squat: opening document part 'f32092'
2022-11-13T01:32:12.083837+01:00 cyrus36 cyrus/squatter[19126]: squat: writing 44 bytes into message 32092
2022-11-13T01:32:12.087780+01:00 cyrus36 cyrus/squatter[19126]: squat: opening document part 't32092'
2022-11-13T01:32:12.088028+01:00 cyrus36 cyrus/squatter[19126]: squat: writing 48 bytes into message 32092
2022-11-13T01:32:12.090078+01:00 cyrus36 cyrus/squatter[19126]: squat: opening document part 's32092'
(red part is suspicious for me)
But I do not find 1:1 between the additional logs and assertion problems: seems more logs than assertion. (Maybe all of them could cause assertion, but squatter normally dies at the first one/account, so, the rest is invisible at this point.)
And unfortunately I was not able to find the exact mails, which caused that.
Using mbexamine, I thought I found some lead, but the theory failed when I tried to prove (emails with similar symptoms could be squattered).
I had no time to modify the squatter sourcecode with some more debug (print the message id/filename) and compiling, yet.
I have the idea to print all filenames which opened for squattering and where the assertion happened, I got the problematic email.
It is still a theory.
Do you think, I could any useful information at this moment?
Thanks,
István
2022. 11. 14, hétfő keltezéssel 10.43-kor ellie timoney ezt írta:
Hi István,I haven't seen a bug report about this come through GitHub yet, did you end up solving the problem yourself?Cheers,ellieOn Tue, 8 Nov 2022, at 11:51 PM, Pongrácz István wrote:Thank you, I check it!cheers,István2022. 11. 8, kedd keltezéssel 12.01-kor Nic Bernstein ezt írta:I would suggest exploring a bit with 'mbexamine(8)' to see what it shows. See the man page here:Specifically, if you have either the UID or the sequence number of a message, you may use the "-u uid" or "-s seqnum" options to narrow down the output.If you then decide to create a bug report, the output from mbexamine(8) will be of use.Cheers,-nicOn 11/8/22 11:55, Sebastian Hagedorn wrote:My suggestion would be to create a bug report on GitHub.
Cheers,SebastianOn 8 Nov 2022, at 12:47, Pongracz Istvan wrote:
Hi,I tried to reconstruct one of the problematic account with this:/usr/lib/cyrus/bin/reconstruct -r -f -R -u pongracziDidn't work, squatter still assert failured:cyrus/squatter[12044]: ERROR: message has more than 1000 header lines not caching any morefatal error: Internal error: assertion failed: imap/squat_internal.c: 134: v64 >= 0In case of 2 out of 7 assertion issues, that more than 1000 header lines exists, but the rest (5) there is no other information, only the folder name available.Any idea?Thank you!IStvánPongrácz István <pongracz.istvan@xxxxxxxxx> ezt írta (időpont: 2022. nov. 7., H, 16:13):Hi,We use the cyrus version 3.6.0-beta3-1+b1 and I found an issue with squatter.In some accounts there are a folder, where the squatter dies with fatal error and the whole squatter process cannot move on.I mean, there are about 40 accounts, called a.... to v....The squatter starts and can finish about 4 accounts, up to user 'foo' where it run into a folder, I assume it found an email and id dies with the following message:process type:EVENT name:squatter1 path:/usr/sbin/cyrus age:138.664s pid:31781 exited, status 70When I issue the command in command line, as cyrus user/usr/lib/cyrus/bin/squatter -v -p -u fooI got the following, more specific error message:Indexing mailbox user/foo/Archives/2013... fatal error: Internal error: assertion failed: imap/squat_internal.c: 134: v64 >= 0As I checked the squatter_internal.c it did not change for years (github).The problem with this, the periodic squatter just dies in the very beginning and other accounts never will be squattered.This server populated using recent imapsync from old cyrus server.The used squatter is: squat (not xapian).Now I have to run squatter "manually", directly for every user to get search function usable, more or less.(using the formula: /usr/lib/cyrus/bin/squatter -v -p -u foo )Do you know any kind of trick or method, how to eliminate/solve this issue?Thank you!István--.:.Sebastian Hagedorn - Weyertal 121 (Gebäude 133), Zimmer 2.13.:..:.Regionales Rechenzentrum (RRZK).:..:.Universität zu Köln / Cologne University - ✆ +49-221-470-89578.:.
