Sent this last night but got caught up for mod approval, switched images for links.
----
Thanks so much for your replies, I really appreciate being able to talk about this. I'm going to give you my full journey!
My normal prod server setup is Debian 10 LXC containers on a slightly older Debian 9 host server running Proxmox 6 at OVH. This is where the problem initially arose last week.
Any Debian 9 container on this server (all running 1.1.0l) connects fine:
...but the Debian 10 containers all fail:
My first thought was can I try to reproduce this on another Debian 10 server elsewhere.
I was able to reproduce this very easily using the Debian 'App' using Windows 10 built-in Windows Subsystem for Linux (WSL) All I did was install the WSL Debian app, apt update/upgrade and apt install openssl:
I've also tried the Ubuntu App on WSL. Out of the box it uses 18.04 LTS and OpenSSL 1.1.1 - It works!
I was suspicious about this '1.1.1 11 Sep 2018', and as I'm not too sure how to upgrade this properly, I thought it was best just to upgrade the Windows App Ubuntu VM to 19.10. This time the version was 1.1.1c 28 May 2019 and is still worked!
Again the OpenSSL version wasn't exactly the same 1.1.1c != 1.1.1d
Not really knowing how to upgrade OpenSSL without using apt, I thought I might as well try to upgrade Ubuntu 20.04 LTS which isn't fully out yet but worth a shot.
Finally, this development version of Ubuntu is running the same release of OpenSSL 1.1.1d and guess what? It failed with the same error:
This is when I thought it was time to jump on the mailing list.
Since you guys both mention it's working for you, I've also just ran this on a brand new bare-metal server I've got for another project hosted at OVH London. This again is Debian 10 / OpenSSL 1.1.1d and has the same result:
So in summary:
openssl s_client -connect thankqcrm.accessacloud.com:443
* Debian 10 + 1.1.1d - Handshake Error
* Debian 9 + 1.1.0l - Working
* Ubuntu 18.04 + 1.1.1
11 Sep 2018 -Working
* Ubuntu 19.10 + 1.1.1c 28 May 2019 - Working
* Ubuntu 20.04 + 1.1.1d - Handshake Error
The handshake errors can be bypassed using switch -tls1_1 on Debian 10
So I'm seeing a pattern here, as for what exactly I'm stumped! If anybody else can replicate using Debian 10 + 1.1.1d I'd appreciate it. But as for where to go from here I'm lost.
Again, thanks for your replies and help.
Niki
On Tue, 10 Mar 2020 at 18:03, Sergio NNX <sfhacker@xxxxxxxxxxx> wrote:
It seems to work fine here!
From: openssl-users <openssl-users-bounces@xxxxxxxxxxx> on behalf of Matt Caswell <matt@xxxxxxxxxxx>
Sent: Wednesday, 11 March 2020 4:25 AM
To: Niki Dinsey <niki.dinsey@xxxxxxxxxxxxxxx>; openssl-users@xxxxxxxxxxx <openssl-users@xxxxxxxxxxx>
Subject: Re: Question about handshake error
On 10/03/2020 17:05, Niki Dinsey wrote:
> Hi there, I have an issue I can't seem to work out the answer to.
>
> Server: thankqcrm.accessacloud.com <http://thankqcrm.accessacloud.com>
>
> root@willis:~# openssl version
> OpenSSL 1.1.1d 10 Sep 2019
> root@willis:~# openssl s_client -connect thankqcrm.accessacloud.com:443
> <http://thankqcrm.accessacloud.com:443>
Running the exact same openssl version from my machine, and using the
exact same s_client options as you, I get a successful connection.
Some possibilities of what might be going wrong:
1) Is it possible there is some middlebox betwen you and the target
server that is causing a problem for you on your network? Can you try
connecting from s_client from a machine outside your corporate network?
or
2) I have sometimes seen load balancers do strange things - where
different machines in the cluster are configured differently to each
other. This can mean different people see different results - or even if
you run the same test at different times you see different results. This
could explain why it works in 1.1.0 and not 1.1.1 (i.e. it actually is
equally broken - but sometimes you hit the "right" server, and sometimes
you hit the "wrong" server). You might want to try from a different
machine and see if the same thing happens, and/or repeat the tests
periodically (in the hope of hitting different servers in the cluster).
or
3) Possibly there is some local problem with your s_client build. Does
it work ok for other sites?
Matt
> CONNECTED(00000004)
> 140151269360768:error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert
> handshake failure:../ssl/record/rec_layer_s3.c:1544:SSL alert number 40
> ---
> no peer certificate available
> ---
> No client certificate CA names sent
> ---
> SSL handshake has read 7 bytes and written 318 bytes
> Verification: OK
> ---
> New, (NONE), Cipher is (NONE)
> Secure Renegotiation IS NOT supported
> Compression: NONE
> Expansion: NONE
> No ALPN negotiated
> Early data was not sent
> Verify return code: 0 (ok)
> ---
>
> Works on OpenSSL 1.1.0:
>
> root@host:~# openssl version
> OpenSSL 1.1.0l 10 Sep 2019
> root@host:~# openssl s_client -connect thankqcrm.accessacloud.com:443
> <http://thankqcrm.accessacloud.com:443>
> CONNECTED(00000003)
> depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com
> <http://www.digicert.com>, CN = DigiCert Global Root CA
> verify return:1
> depth=1 C = US, O = DigiCert Inc, OU = www.digicert.com
> <http://www.digicert.com>, CN = Thawte RSA CA 2018
> verify return:1
> depth=0 CN = *.accessacloud.com <http://accessacloud.com>
> verify return:1
> ---
> Certificate chain
> 0 s:/CN=*.accessacloud.com <http://accessacloud.com>
> i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=Thawte
> <http://www.digicert.com/CN=Thawte> RSA CA 2018
> 1 s:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=Thawte
> <http://www.digicert.com/CN=Thawte> RSA CA 2018
> i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert
> <http://www.digicert.com/CN=DigiCert> Global Root CA
> 2 s:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert
> <http://www.digicert.com/CN=DigiCert> Global Root CA
> i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert
> <http://www.digicert.com/CN=DigiCert> Global Root CA
> 3 s:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert
> <http://www.digicert.com/CN=DigiCert> Global Root CA
> i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert
> <http://www.digicert.com/CN=DigiCert> Global Root CA
> ---
> Server certificate
> -----BEGIN CERTIFICATE-----
> ...
> -----END CERTIFICATE-----
> subject=/CN=*.accessacloud.com <http://accessacloud.com>
> issuer=/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=Thawte
> <http://www.digicert.com/CN=Thawte> RSA CA 2018
> ---
> No client certificate CA names sent
> ---
> SSL handshake has read 4999 bytes and written 494 bytes
> Verification: OK
> ---
> New, TLSv1.2, Cipher is AES128-GCM-SHA256
> Server public key is 2048 bit
> Secure Renegotiation IS supported
> Compression: NONE
> Expansion: NONE
> No ALPN negotiated
> SSL-Session:
> Protocol : TLSv1.2
> Cipher : AES128-GCM-SHA256
> Session-ID:
> 05326CD4A0D128684EA530A59504BA8D02E99746AC2E40D0DA8B9B0E18F20CF0
> Session-ID-ctx:
> Master-Key:
> B423C27867FFB6A021458D860CC8A5A6D947628A8216B5F8DD8D1CF3058545398185B94F772B3A816A15D1442FFF1822
> PSK identity: None
> PSK identity hint: None
> SRP username: None
> TLS session ticket lifetime hint: 14400 (seconds)
> TLS session ticket:
> 0000 - e5 7b cf ea bc 3d 6b 9a-59 ec 40 63 01 19 52 6c
> .{...=k.Y.@xxxxx
> 0010 - 72 c4 34 f0 a3 ff 37 f4-58 b1 9a bb 84 fc 94 36
> r.4...7.X......6
> 0020 - 16 8e 39 04 94 e2 fd ae-0f 05 e7 6c 12 94 58 4a
> ..9........l..XJ
> 0030 - 09 56 e5 bd 67 d7 e7 17-d4 a8 03 ba 6e 05 be b6
> .V..g.......n...
> 0040 - ce 5d 9a ee 81 73 97 c8-ff 9c be 6b 8f 37 cb bf
> .]...s.....k.7..
> 0050 - 44 76 93 83 95 58 6d b8-63 f6 ba 4d 55 22 d2 14
> Dv...Xm.c..MU"..
> 0060 - 93 09 01 46 f0 fa f1 35-5a 80 0e ab a4 ca 9e c8
> ...F...5Z.......
> 0070 - ed 8f c8 3c 89 e8 91 b3-0e 41 a9 e4 3f 79 f6 63
> ...<.....A..?y.c
> 0080 - e2 62 91 c9 2f 8c 5a dd-b0 a1 55 b3 86 35 62 5a
> .b../.Z...U..5bZ
> 0090 - af c2 9a 8a 35 7a 46 3b-3c 2e 24 0d 45 69 96 fc
> ....5zF;<.$.Ei..
>
> Start Time: 1583859230
> Timeout : 7200 (sec)
> Verify return code: 0 (ok)
> Extended master secret: no
> ---
>
>
> Works using 1.1.1d if I pass in -tls1_1
>
> root@willis:~# openssl version
> OpenSSL 1.1.1d 10 Sep 2019
> root@willis:~# openssl s_client -connect thankqcrm.accessacloud.com:443
> <http://thankqcrm.accessacloud.com:443> -tls1_1
> CONNECTED(00000004)
> depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com
> <http://www.digicert.com>, CN = DigiCert Global Root CA
> verify return:1
> depth=1 C = US, O = DigiCert Inc, OU = www.digicert.com
> <http://www.digicert.com>, CN = Thawte RSA CA 2018
> verify return:1
> depth=0 CN = *.accessacloud.com <http://accessacloud.com>
> verify return:1
> ---
> Certificate chain
> 0 s:CN = *.accessacloud.com <http://accessacloud.com>
> i:C = US, O = DigiCert Inc, OU = www.digicert.com
> <http://www.digicert.com>, CN = Thawte RSA CA 2018
> 1 s:C = US, O = DigiCert Inc, OU = www.digicert.com
> <http://www.digicert.com>, CN = Thawte RSA CA 2018
> i:C = US, O = DigiCert Inc, OU = www.digicert.com
> <http://www.digicert.com>, CN = DigiCert Global Root CA
> 2 s:C = US, O = DigiCert Inc, OU = www.digicert.com
> <http://www.digicert.com>, CN = DigiCert Global Root CA
> i:C = US, O = DigiCert Inc, OU = www.digicert.com
> <http://www.digicert.com>, CN = DigiCert Global Root CA
> 3 s:C = US, O = DigiCert Inc, OU = www.digicert.com
> <http://www.digicert.com>, CN = DigiCert Global Root CA
> i:C = US, O = DigiCert Inc, OU = www.digicert.com
> <http://www.digicert.com>, CN = DigiCert Global Root CA
> ---
> Server certificate
> -----BEGIN CERTIFICATE-----
> ...
> -----END CERTIFICATE-----
> subject=CN = *.accessacloud.com <http://accessacloud.com>
>
> issuer=C = US, O = DigiCert Inc, OU = www.digicert.com
> <http://www.digicert.com>, CN = Thawte RSA CA 2018
>
> ---
> No client certificate CA names sent
> ---
> SSL handshake has read 5059 bytes and written 481 bytes
> Verification: OK
> ---
> New, SSLv3, Cipher is AES128-SHA
> Server public key is 2048 bit
> Secure Renegotiation IS supported
> Compression: NONE
> Expansion: NONE
> No ALPN negotiated
> SSL-Session:
> Protocol : TLSv1.1
> Cipher : AES128-SHA
> Session-ID:
> 9438801392B268A70F6B60C25E388481D69638ED8122A274BB0E15111BFF329B
> Session-ID-ctx:
> Master-Key:
> EA86A4D07020F193BC66444A2D16EC67AD9524A6A78D068542B6CAF745D0B51FBE51EA0F9E9A6557561CFD5AE9E2D986
> PSK identity: None
> PSK identity hint: None
> SRP username: None
> TLS session ticket lifetime hint: 14400 (seconds)
> TLS session ticket:
> 0000 - e5 7b cf ea bc 3d 6b 9a-59 ec 40 63 01 19 52 6c
> .{...=k.Y.@xxxxx
> 0010 - 3a c0 bc fb ff 57 a2 7f-38 a9 91 64 5e 87 b4 88
> :....W..8..d^...
> 0020 - f2 35 bc 04 b3 27 b3 fc-0f ac 3d 8a 03 a4 59 cb
> .5...'....=...Y.
> 0030 - a7 2c 8e 0f f3 a0 a2 13-50 fa 6f 2e 07 eb 1e 89
> .,......P.o.....
> 0040 - 73 0d d0 3e d5 01 68 3a-18 56 00 71 fa 38 1e e0
> s..>..h:.V.q.8..
> 0050 - 87 15 68 a4 d0 d7 13 67-c7 b1 e6 45 54 fd 22 e1
> ..h....g...ET.".
> 0060 - 65 66 40 6c e3 7e 42 f1-1a 46 32 7b b9 a1 c0 80
> ef@l.~B..F2{....
> 0070 - 12 ee f1 d9 92 5f b7 3b-7b 38 66 76 cc af b1 eb
> ....._.;{8fv....
> 0080 - 97 4c 02 af 61 9d 1b 35-c8 64 f5 ce 19 34 42 92
> .L..a..5.d...4B.
> 0090 - a0 0e b9 51 ab de c0 cf-90 bd 65 2b 0b 08 19 3b
> ...Q......e+...;
> 00a0 - 2e fe 1f 75 1f b5 b8 48-40 8c 56 d4 dc 82 31 b0
> ...u...H@.V...1.
> 00b0 - 2f 52 b9 1f 11 f7 d2 63-01 c0 89 57 dd a6 53 56
> /R.....c...W..SV
>
> Start Time: 1583859354
> Timeout : 7200 (sec)
> Verify return code: 0 (ok)
> Extended master secret: no
> ---
>
>
> ---------------------------------
> This error started our of the blue. The vendor confirmed a change in
> certificate about the same time that curl/'python requests' stopped
> working. So it looks to me like their cert change caused the issue.
>
> Tested on Debian 10 and Ubuntu 20.04 Focal Fossa.
>
> Why does this certificate work with tls1.2 on 1.1.0 but not on 1.1.1???
>
> I can force tls1.1, but want to inform the vendor there are problems
> with their new certificate but don't really understand much of this.
>
> Any help appreciated.
>
> Regards
>
> Niki
>
>
> Save the date: Abingdon's first 24hr *Giving Day - 18 March 2020*.
> Help support our ambition to double the number of bursaries across the
> Foundation.
>
> <http://www.150givingday.abingdon.org.uk>
>
>
> Abingdon School: A company limited by guarantee Registered in England
> and Wales. Company No. 3625063
>
> Registered Office:
> Abingdon School
> Park Road
> Abingdon
> OX14 1DE
> Registered Charity No. 1071298
>
> All information in this message and attachments is confidential and may
> be legally privileged. Only intended recipients are authorised to use
> it. E-mail transmissions are not guaranteed to be secure or error free
> and the sender does not accept liability for such errors or omissions.
> The company will not accept any liability in respect of such
> communication that violates our ICT policies.
Niki Dinsey
IS Manager
07974 214718
01235 849061 (x261)
Save the date: Abingdon's first 24hr Giving Day - 18 March 2020.
Help support our ambition to double the number of bursaries across the Foundation.
![]()
Abingdon School: A company limited by guarantee Registered in England and Wales. Company No. 3625063
Registered Office:
Abingdon School
Park Road
Abingdon
OX14 1DE
Registered Charity No. 1071298
All information in this message and attachments is confidential and may be legally privileged. Only intended recipients are authorised to use it. E-mail transmissions are not guaranteed to be secure or error free and the sender does not accept liability for such errors or omissions. The company will not accept any liability in respect of such communication that violates our ICT policies.
