I feel like I am getting somewhere.
If I change the search base back to dc=MyCompany,dc=local in imapd.conf (matching saslauthd), when I run imtest with pts enabled I do get searchResRef stanzas back from the server. These represent other branches of the directory.
Lightweight Directory Access Protocol
LDAPMessage searchResEntry(2) "CN=Test
User,OU=SBSUsers,OU=Users,OU=MyBusiness,DC=MyCompany,DC=local" [1
result]
messageID: 2
protocolOp: searchResEntry (4)
searchResEntry
objectName: CN=Test
User,OU=SBSUsers,OU=Users,OU=MyBusiness,DC=MyCompany,DC=local
attributes: 0 items
[Response To: 679124]
[Time: 0.000733000 seconds]
Lightweight Directory Access Protocol
LDAPMessage searchResRef(2)
messageID: 2
protocolOp: searchResRef (19)
searchResRef: 1 item
LDAPURL:
ldap://ForestDnsZones.MyCompany.local/DC=ForestDnsZones,DC=MyCompany,DC=local
[Response To: 679124]
[Time: 0.000733000 seconds]
Lightweight Directory Access Protocol
LDAPMessage searchResRef(2)
messageID: 2
protocolOp: searchResRef (19)
searchResRef: 1 item
LDAPURL:
ldap://DomainDnsZones.MyCompany.local/DC=DomainDnsZones,DC=MyCompany,DC=local
[Response To: 679124]
[Time: 0.000733000 seconds]
Lightweight Directory Access Protocol
LDAPMessage searchResRef(2)
messageID: 2
protocolOp: searchResRef (19)
searchResRef: 1 item
LDAPURL:
ldap://MyCompany.local/CN=Configuration,DC=MyCompany,DC=local
[Response To: 679124]
[Time: 0.000733000 seconds]
Lightweight Directory Access Protocol
LDAPMessage searchResDone(2) success [1 result]
messageID: 2
protocolOp: searchResDone (5)
searchResDone
resultCode: success (0)
matchedDN:
errorMessage:
[Response To: 679124]
[Time: 0.000733000 seconds]
When I look at the searches (and wireshark hung so I had to start a new session to get these) I see:
Pts enabled:
Lightweight Directory Access Protocol
LDAPMessage searchRequest(2) "dc=MyCompany,dc=local"
wholeSubtree
messageID: 2
protocolOp: searchRequest (3)
searchRequest
baseObject: dc=MyCompany,dc=local
scope: wholeSubtree (2)
derefAliases: neverDerefAliases (0)
sizeLimit: 1
timeLimit: 5
typesOnly: False
Filter: (sAMAccountName=testuser)
filter: equalityMatch (3)
attributes: 1 item
AttributeDescription: 1.1
[Response In: 588]
PTS disabled:
Actually, this is coming through, saslauthd must have cached the last search and is authenticating without contacting the server, but I'm pretty sure the only difference in the search is that saslauthd is looking for attribute "dn" where ptload is looking for "1.1"
Whatever ldap_member_attribute I set, the search always has AttributeDescription: 1.1 so is it an issue with the attribute that ptload is requesting? I need to go through the man page for imapd.conf again and see if i have missed something that defines an attribute which is overriding ldap_user_attribute, ldap_group_attribute and ldap_member_attribute.
Thanks
Jim
Hi Andrew,
If I am understanding the packet logs correctly, the LDAP server is responding (only inserting the ldap parts of the log entry):
The search:
Lightweight Directory Access Protocol
LDAPMessage searchRequest(2) "ou=MyBusiness,dc=MyCompany,dc=local" wholeSubtree
messageID: 2
protocolOp: searchRequest (3)
searchRequest
baseObject: ou=MyBusiness,dc=MyComapny,dc=local
scope: wholeSubtree (2)
derefAliases: neverDerefAliases (0)
sizeLimit: 1
timeLimit: 5
typesOnly: False
Filter: (sAMAccountName=testuser)
filter: equalityMatch (3)
equalityMatch
attributeDesc: sAMAccountName
assertionValue: testuser
attributes: 1 item
AttributeDescription: 1.1
[Response In: 557126]The results:
Lightweight Directory Access Protocol
LDAPMessage searchResEntry(2) "CN=Test User,OU=SBSUsers,OU=Users,OU=MyBusiness,DC=MyCompany,DC=local" [1 result]
messageID: 2
protocolOp: searchResEntry (4)
searchResEntry
objectName: CN=Test User,OU=SBSUsers,OU=Users,OU=MyBusiness,DC=MyCompany,DC=local
attributes: 0 items
[Response To: 557125]
[Time: 0.047382000 seconds]
Lightweight Directory Access Protocol
LDAPMessage searchResDone(2) success [1 result]
messageID: 2
protocolOp: searchResDone (5)
searchResDone
resultCode: success (0)
matchedDN:
errorMessage:
[Response To: 557125]
[Time: 0.047382000 seconds]I did change my ldap base from dc=MyCompany,dc=local when I noticed it seemed to be querying additional parts of the directory, although I am quite sure all users are unique. It is now:
ldap_base: ou=MyBusiness,dc=MyCompany,dc=local
With the group and member bases set to DistributionGroups:
ldap_group_base: ou=DistributionGroups,ou=MyBusiness,dc=MyCompany,dc=local
ldap_member_base: ou=DistributionGroups,ou=MyBusiness,dc=MyCompany,dc=local
The LDAP is still mostly laid out as per the MS Small Business Server 2003 way of setting up the schema (but on 2012 R2), I have no idea if it is normal for an AD installation. The distribution groups ou is one I added to try to keep my distribution groups away from the security groups.
Anyway, going back to the results, I don't understand this at low level but you are asking about searchresref and the server seems to be giving searchResEntry - is this as simple as MS implementing non-standard methods?
But, in searchResEntry, whilst it finds the objectName, which is a DN, it is returning 0 attributes, so it can't be as simple as ptloader doesn't understand the response, the response also appears incomplete? (I have ldap_member_attribute: set and have tried all sorts of attributes that I know are in the directory, including sAMAccoutName which we can see it recognises in the search itself).
I already set the debug option on ptloader (forgot to say in first post) in cyrus.conf:
ptloader cmd="ptloader -d1" listen="var/lib/cyrus/ptclient/ptsock"
And have tried increasing from 1 but as others have reported it does not seem to affect verbosity. The debug output goes in syslog similar to the lines I had before although now that I at least have the bind working I now get:
cyrus/ptloader[63125]: searching ou=MyBusiness,dc=MyCompany,dc=local with (sAMAccountName=testuser) failed
Now, what is interesting and needs more investigation later, is that if I disable pts again and use the same imtest command, it first preforms a bind using the saslauthd user (so I know sasluthd is making the bind), the results do include searchResRef entries (several, I think because I am searching a at dc level?) and cyrus goes on to make a subsequent bind as testuser, using testuser's password (in the pts examples the initial bind is by the cyrus user, and there is no subsequent bind for the test user).
I need to make a site visit now so will dig into this some more later.
thanks for your help so far!
Jim
On 16/06/2021 18:45, AndrewHardy via Info wrote:Hi Jim,It sounds like you’ve spotted a few issues and have fixed them. I’m quite interested to know what your imapd configuration is for pts and also how the configuration knows what OU the user you’re testing resides in? I haven't personally set up LDAP auth with Cyrus although it’s been something I’ve been wanting to do for some time (so showing a bit of interest in this thread).When you mentioned nothing seems to get returned to ptloader, is it possible to confirm if there was a ldap response from the user search in the ldap packet trace?Was the result count returned and what was the format of searchresref? If that is returned on the wire (confirming user was found after the initial search) then I guess next step is to debug ptloader to see what that’s expecting to receive and whether the ldap response data gets there ? I’m assuming the search does find the user of interest and the result count is 1 and the distinguished name of the object is returned over the wire to the mail server? If the user is searched but not found, that’s likely a place to investigate e.g where in the tree is it trying to find the user - does the attribute you’re checking for and the user searched match up?For ptloader, there is a debug option. May be worth enabling that although not sure what that technically offers in terms of troubleshooting info. Im a bit unsure on how to attach strace in this context, which process ptloader operates under or whether it’s it’s own seperate process with its own pid. If it’s the later it’s probably going to be a bit easier. Bit of trial and error most likely.RegardsAndrewSent from my iPhoneOn 17/06/2021, at 03:48, Jim Wallis <jwallis@xxxxxxxxxxxxxxxxx> wrote:Hi Andrew,
This has been very helpful. I didn't think of using wireshark before, it has enabled me to at least sort out a typo in my bind DN , but I am still not much further on.
With auth_mech: pts commented out, and using an imtest line like:
imtest -u test -a test -w passfortest localhost
I can see from the wireshark capture that Saslauthd is binding as the test user and imtest authenticates OK
with auth_mech: pts uncommented I need to provide bind credentials in imapd.conf (I use the ones for my cyrus user so I can tell what is being passed). This cyrus user binds successefully and looks up the test user, but nothing seems to get returned to ptloader, so I probably need to spend more time thinking about what attribute I need to return (I have tried several likely candidates). It doesn't look as though saslauthd is used at all when I have pts on. Imtest responds not authenticated.
More work to do... I guess I need to learn how to use strace.
Thanks
Jim
On 15/06/2021 20:30, AndrewHardy via Info wrote:Hi Jim,I’m not sure specifically what your issue is here but I’ll help provide some options for debugging ldap (specific to the windows logging question). I’d suggest dropping the TLS 636 LDAPS and just start with standard LDAP 389, removing TLS complexity whilst you validate a working configuration.I’d suggest also just running wireshark on the domain controller where you expect ldap connections to reach and capture the 389 traffic with a filter of your source mail server (or component doing the authentication). You’d be able to assess the ldap flow; whether the server sees it or not (and what the response was from the server, if any).You could also open another terminal, open saslauthd with strace and dump the authentication process to file to analyse what may be happening. If you can’t wireshark the target authentication service, you could perhaps tcpdump on the ldap client side, drop that into a .pcap file then open this in wireshark on your desktop for further analysis.I’m hoping those potential options can help verify what is/isn’t happening. An absence of traffic leaving the server would imply something on the client side, ruling out the ldap server at fault - at least initially.Hope this helps simplify troubleshooting and helps confirm what is/isn’t happening at least from an ldap perspective. Unsure of enabling ldap debugging in windows, I’ve searched for that for years and never really come up with a good solution. Wireshark and/or tcpdump has helped me solve all problems I’ve had with ldap/authentication although ldaps can be a bit more problematic when just using packet tracing software. Can still come in useful though.RegardsAndrewSent from my iPhoneOn 16/06/2021, at 06:30, jwallis@xxxxxxxxxxxxxxxxx wrote:It seemed like it would be a good idea to use the existing Active Directory DC as the LDAP source for all mail users.I got cyrus (3.2.6 from Buster backports) running using saslauthd for authentication against the directory, and test users could authenticate OK and see their mailbox in Thunderbird. relevant entires in imapd.conf:allowplaintext: yessasl_mech_list: PLAINsasl_pwcheck_method: saslauthdBecause I want to use some groups for shared folders and distribution lists, I also want to authorize users and groups against the directory so enabled:auth_mech: ptspts_module: ldapptloader_sock: var/lib/cyrus/ptclient/ptsockldap_uri: ldaps:/companydomaincontroller:636ldap_bind_dn: a cn that works in other bindsldap_password: the password for aboveldap_sasl: 0ldap_version: 3ldap_ca_dir: /etc/ssl/certs (which is where the ca cert that works with saslauthd resides)ldap_verify_peer: 1ldap_base: base that other binds can successfully find users fromldap_scope: subldap_filter: (sAMAccountName=%u) (I have tried more complex filters and decided on this as one that should work)ldap_user_attribute: mailldap_size_limit: 1024I have commented out the group member stuff for now, need to walk before I can run!With these options enabled, no one can authenticate, even though my understanding is that authentication is distinct from authorization. Thunderbird users can no longer login and imtest for user cyrus (which is in the directory) gives the same output up to a line :C: A01 AUTHENTICATE PLAIN {a hash}S: A01 NO authentication failureAuthentication failed. generic failureSecurity strength factor: 256(the SSF is reported the same, but I have for now only asked for level 0)In /var/log/syslog I find the following lines:cyrus/ptloader[8230]: ldap_initialize failed (ldaps:/DC:636)cyrus/imap[8229]: timeout_select exiting. r = 1; errno = 0cyrus/imap[8229]: timeout_select: sock = 11, rp = 0x7ffecbb6ad30, wp = 0x0, sec = 30cyrus/imap[8229]: timeout_select exiting. r = 1; errno = 0cyrus/imap[8229]: ptload read data backcyrus/imap[8229]: ptload(): bad response from ptloader server: ptsmodule_connect() failedcyrus/imap[8229]: No data available at all from ptload()cyrus/imap[8229]: ptload completely failed: unable to canonify identifer: cyruscyrus/imap[8229]: SASL bad userid authenticatedcyrus/imap[8229]: badlogin: localhost [::1] PLAIN (-notset-) [SASL(-6): can't request information until later in exchange: Information that was requested is not yet available.]I have been searching for answers for days and at one point found a reference that claimed ptsloader is not enabled by default in Debian, so I have downloaded the source package and compiled cyrus-imaps using a configure script based on the Debain default config options with some extra options: --with-auth=pts --with-pts=ldap and --with-ldapThis has made no difference.I have also downlaoded the 3.4.1 source package from experimental and compiled with the same options ands still no difference to behaviour so suspect this is a red herring?So why is ptloader not retrieving any data?The ldap_bind credentials I have given it work fine with saslauthd or postfix or ldapsearch.Presumably then my filter and user attributes are bad? But I can't see why.What is the order of operations within cyrus?I assume that it authenticates first using saslauthd, and then uses the same username to check authorization in pts, but enabling PTS seems to prevent authentication. Although in syslog it is suggesting that SASL has authenticated, but with a bad userid?The windows DC uses a directory migrated from an older one on a small business server where microsoft recommended using an internal .local domain which has always been a bit of a headache for me. This means that our search base is a DC=local, as are the bind DN and userPrincipleName, but the mail and proxyAddresses email addresses are all .comIs the problem related to this? do I need to enable virtual domains and/or cross realm authentication for ptloader to get a response from the server?Is there any way to call ptloader outside of master to try to work out exactly what is being passed and what result it achieves?Also, how does ptdump work? I get no indication that it has done anything, is this simply because ptloader has never yet obtained any data for it to dump?As for checking the LDAP server logs, if anyone knows how I can do this on windows 2012 please advise! Looking at directory services in event viewer I see very few entries and nothing relating to communication from my mail server. I assume I need to enable a different log level but I can't find out how or what.All the examples I can find are based on openldap installations, is what I am trying to do possible, or are the Active Directory schema completely incompatible with ptloader?Jim Wallis
