Re: Can't view (other) users subfolders with imapd 3.0.6

Bron Gondwana <brong@xxxxxxxxxxxxxxxx> · Thu, 07 Jun 2018 04:47:47 +1000

All the namespace conversion is supposed to be encapsulated in a couple of places. This breaks that, which means that something earlier of later is doing its own version of pattern to mailbox conversion. Ideally this would be captured in the abstraction layer in mbname. I haven't looked deeply though.



It's correct, but it likely points to an abstraction leakage elsewhere.


Bron.




On Wed, Jun 6, 2018, at 13:47, ellie timoney wrote:

On Tue, Jun 5, 2018, at 7:27 PM, Edda wrote:

thank you very much for the handling of my patch! I don't have an
      account on github. So, it's easiest for me to just dump it on the
      users list. Please let me know if that bothers you.






It's fine for small patches like this, especially when they work right first time. ;)



For larger pieces of work, a pull request certainly makes it easier to collaboratively iterate towards a good solution!  But if you'd prefer to not create a GitHub account for whatever reason, that's fair enough.



As an author, "Edda" would be fine :) Thank you.






Great!  This is now on the 3.0 and master branches.  Thanks very much! :)



On Tue, Jun 5, 2018, at 8:24 PM, Bron Gondwana wrote:

The patch looks correct at a reading.  It also looks like it's masking a deeper bug, but that's fine - we can fix that later :)






What are you seeing that I'm missing?



I can see that this bug was not just "other users", but any common/solo list pattern with a hiersep in it, but this patch is general enough to fix both.





Best regards,

Edda



Am 05.06.18 um 04:50 schrieb ellie timoney:


Hi Edda,



Thanks for the patch!  I've added your email below to the
        GitHub issue for this: https://github.com/cyrusimap/cyrus-imapd/issues/2362



I'll review the patch and if it looks good, will include it
        in the next 3.0 release.  I'll also forward-port it to master if
        necessary.



Let me know if you'd like to submit this as a Pull Request,
        so your GitHub identity can be credited for it, otherwise I'll
        just set your email address as the author on the commit. :)



Cheers,



ellie





On Mon, Jun 4, 2018, at 9:51 PM, Edda wrote:

Hi,



I was able to reproduce the issue with 3.0.5 with these
            settings:



unixhierarchysep: yes

altnamespace: no

crossdomains: no



Key factor is unixhierarchysep on. (crossdomains is a
            special case. altnamespace on/off only looks different.)



I did a lot of debugging and found out the following:



Compared to version 2.4.x and 2.5.x 3.0.x function
            list_data in ../imap/imapd.c does not transform the
            hierarchy separator.



In function mboxlist_do_find (../imap/mboxlist.c) below
            "Other Users namespace" we have a prefix search. With
            "unixhierarchysep on" cyrus uses the wrong prefix
            (domainpat) for that search.



Example:



. LIST "" "user/%"

* LIST (\HasChildren) "/" user/bbb

. OK Completed (0.000 secs 4 calls)

. LIST "" "user/bbb/sub01"

. OK Completed (0.000 secs)



In this case cyrus searches with domainpat:
            user.bbb/sub01/subsub01



The prefix is previously generated from a variable
            called commonpat, which is used only for this specific
            search.



So I fixed it there with this patch:

--- cyrus-imapd-3.0.5/imap/mboxlist.c.orig    2018-04-12
            13:12:47.633956338 +0200

+++ cyrus-imapd-3.0.5/imap/mboxlist.c    2018-05-30
            10:56:44.255724710 +0200

@@ -2774,6 +2774,10 @@

             const char *pat = strarray_nth(patterns,
            i);

             if (pat[prefixlen] != c) break;

         }

+        if (rock->namespace->hier_sep == '/') {

+            if (c == '/') c = '.';

+            else if (c == '.') c = DOTCHAR;

+        }

         if (i < patterns->count) break;

         if (c == '*' || c == '%' || c == '?') break;

         commonpat[prefixlen] = c;





Now for the example above the search pattern is
            user.bbb.sub01 and the submailbox is returned correctly.



. LIST "" "user/%"

* LIST (\HasChildren) "/" user/bbb

. OK Completed (0.000 secs 4 calls)

. LIST "" "user/bbb/sub01"

* LIST (\HasChildren) "/" user/bbb/sub01

. OK Completed (0.000 secs 2 calls)

. LIST "" "user/bbb/%"

* LIST (\HasChildren) "/" user/bbb/sub01

* LIST (\HasNoChildren) "/" user/bbb/sub.dot01

. OK Completed (0.000 secs 3 calls)

. SUBSCRIBE "user/bbb/sub01"

. OK Completed

. LSUB "" "user/*"

* LSUB () "/" user/bbb/sub01

. OK Completed (0.000 secs 1 calls)



The patch works fine in my test environment and I see no
            caveats yet. I would be very nice if someone could check it
            :-)



Best regards,

Edda









Am 15.05.18 um 10:37 schrieb marco@xxxxxxxxxx:


Hi,
after migrating from cyrus-imapd-2.4.17 to cyrus-imapd-3.0.6 we cannot
view (other) users subfolders.

Distro:Gentoo
Cyrus version: cyrus-imapd-3.0.6

imapd-3.0.5.conf attached

In the test environment we created a user named "aaa" and "bbb", then
we created the folder "bbb/mytest/mysubtest". We set the acl for the
user aaa to share the mailbox bbb/mytest/mysubtest

user.aaa        0 default aaa   lrswipkxtecdan  anonymous       p
user.bbb        0 default bbb   lrswipkxtecdan  anonymous       p
	aaa     lrswipkxtecdan
user.bbb.mytest 0 default bbb   lrswipkxtecdan  anonymous       p
	aaa     lrswipkxtecdan
user.bbb.mytest.mysubtest       0 default 
	bbb   lrswipkxtecdan
	anonymous       p       
	aaa     lrswipkxtecdan

>From whatever imap client, when we search for new folders, user.bbb
appears, but neither user.bbb.mytest nor user.bbb.mytest.mysubtest.

This is the capture from imapd 3.0.6:

IMAP> 62 LIST "" "user/%" 
IMAP< * LIST (\HasChildren) "/" user/bbb 
IMAP< 62 OK Completed (0.001 secs 3 calls) 
IMAP> 63 LIST "" "user/bbb/%" 
IMAP< 63 OK Completed (0.000 secs) 
IMAP> 64 LSUB "" "user/bbb/%" 
IMAP< 64 OK Completed (0.000 secs) 
IMAP> 65 LSUB "" "user/%" 
IMAP< 65 OK Completed (0.000 secs) 
IMAP> 66 LSUB "" "%" 
IMAP< * LSUB (\Noselect \HasChildren) "/" INBOX 
IMAP< 66 OK Completed (0.000 secs 4 calls) 
IMAP> 16 NOOP 
IMAP< 16 OK Success 
IMAP> 67 SUBSCRIBE "user/bbb" 
IMAP< 67 OK Completed 
IMAP> 68 LSUB "" "%" 
IMAP< * LSUB (\Noselect \HasChildren) "/" INBOX 
IMAP< * LSUB (\Noselect \HasChildren) "/" user 
IMAP< 68 OK Completed (0.001 secs 5 calls) 
IMAP> 69 LSUB "" "INBOX/%" 
IMAP< * LSUB () "/" INBOX/Drafts 
... 
IMAP> 74 LSUB "" "user/%" 
IMAP< * LSUB () "/" user/bbb 
IMAP< 74 OK Completed (0.001 secs 1 calls) 
IMAP> 75 LSUB "" "user/bbb/%" 
IMAP< 75 OK Completed (0.001 secs)

Refreshing only the  user.bbb folders nothing changed

IMAP> 83 LIST "" "user/%" 
IMAP< * LIST (\HasChildren) "/" user/bbb 
IMAP< 83 OK Completed (0.001 secs 3 calls) 
IMAP> 84 LIST "" "user/bbb/%" 
IMAP< 84 OK Completed (0.001 secs) 
IMAP> 85 LSUB "" "user/bbb/%" 
IMAP< 85 OK Completed (0.001 secs) 
IMAP> 86 LSUB "" "user/%" 
IMAP< * LSUB () "/" user/bbb 
IMAP< 86 OK Completed (0.001 secs 1 calls)

With net-mail/cyrus-imapd-2.4.17 all is working fine 

imapd-2.4.17.conf attached

user.aaa        0 default aaa   lrswipkxtecda   anonymous       p
user.bbb        0 default bbb   lrswipkxtecda   anonymous       p
	aaa     lrswipkxtecda
user.bbb.mytest 0 default bbb   lrswipkxtecda   anonymous       p
	aaa     lrswipkxtecda
user.bbb.mytest.mysubtest       0 default bbb   lrswipkxtecda
	anonymous       p
	aaa     lrswipkxtecda

if we search for new folders to subscribe user/bbb and all the
subfolders appeared

IMAP> 57 LIST "" "user/%" 
IMAP< * LIST (\HasChildren) "/" user/bbb 
IMAP< 57 OK Completed (0.000 secs 3 calls) 
IMAP> 58 LIST "" "user/bbb/%" 
IMAP< * LIST (\HasChildren) "/" user/bbb/mytest 
IMAP< 58 OK Completed (0.000 secs 2 calls) 
IMAP> 59 LIST "" "user/bbb/mytest/%" 
IMAP< * LIST (\HasNoChildren) "/" user/bbb/mytest/mysubtest 
IMAP< 59 OK Completed (0.000 secs 2 calls) 
IMAP> 60 LIST "" "user/bbb/mytest/mysubtest/%" 
IMAP< 60 OK Completed (0.000 secs) 
IMAP> 61 LSUB "" "user/bbb/mytest/mysubtest/%" 
IMAP< 61 OK Completed (0.000 secs) 
IMAP> 62 LSUB "" "user/bbb/mytest/%" 
IMAP< 62 OK Completed (0.000 secs) 
IMAP> 63 LSUB "" "user/bbb/%" 
IMAP< 63 OK Completed (0.000 secs) 
IMAP> 64 LSUB "" "user/%" 
IMAP< 64 OK Completed (0.000 secs) 
IMAP> 65 LSUB "" "%" 
IMAP< * LSUB (\Noselect \HasChildren) "/" INBOX 
IMAP< 65 OK Completed (0.000 secs 4 calls) 
IMAP> 66 NOOP 
IMAP< 66 OK Completed 
IMAP> 67 SUBSCRIBE "user/bbb" 
IMAP< 67 OK Completed 
IMAP> 68 SUBSCRIBE "user/bbb/mytest/mysubtest" 
IMAP< 68 OK Completed 
IMAP> 69 SUBSCRIBE "user/bbb/mytest" 
IMAP< 69 OK Completed

Any suggestion?

Thanks for the support
Marco







----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus


----

Cyrus Home Page: http://www.cyrusimap.org/

List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

To Unsubscribe:

https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus







----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

----

Cyrus Home Page: http://www.cyrusimap.org/

List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

To Unsubscribe:

https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus



----

Cyrus Home Page: http://www.cyrusimap.org/

List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

To Unsubscribe:

https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus



--

  Bron Gondwana, CEO, FastMail Pty Ltd

  brong@xxxxxxxxxxxxxxxx






----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus



