self-signed certificate won't work in my app but works with s_client

[cheyendal@xxxxxxxxxxxx (Carl Heyendal)]

I am working with the example apps in the "Networking Security with OpenSSL" book and up until now have been able to get client/server examples 1,2,3 to work. But now I'm trying to connect to an in-house tool but I'm getting the error "error 18:self signed certificate". Despite this error when I run my app (essentially client3), when I use s_client with the very same credentials...it works.

I suspect that it has something to do with the ssl/tls api combination that I use in my 'client3' app.

Here's the command and output for s_client that connects to the in-house tool which works:

    > openssl s_client -connect 192.168.1.99:16001 -CAfile ../_security/SipInspector/certificate.pem -key ../_security/client.pem
      Enter pass phrase for ../_security/client.pem:
      CONNECTED(00000003)
      depth=0 C = CA, ST = Ontario, L = Ottawa, O = SIP Inspector Ltd, OU   =     Development, CN = 192.168.1.99
      verify return:1
      ---
      Certificate chain
       0 s:/C=CA/ST=Ontario/L=Ottawa/O=SIP Inspector Ltd/OU=Development/CN=192.168.1.99
         i:/C=CA/ST=Ontario/L=Ottawa/O=SIP Inspector Ltd/OU=Development/CN=192.168.1.99
      ---
      Server certificate
      -----BEGIN CERTIFICATE-----
      MIIFxTCCA62gAwIBAgIJALKQ3J5SEyjPMA0GCSqGSIb3DQEBCwUAMHkxCzAJBgNV
      BAYTAkNBMRAwDgYDVQQIDAdPbnRhcmlvMQ8wDQYDVQQHDAZPdHRhd2ExGjAYBgNV
	(snip)
      pt/q5/gKqRFbjyL0LDNz49vaSUYvbu3mgF2480Or4X+GPwemwdxJaF1pQw4C1WaF
      RyfVjDrLNhTvv+zKCbEPyrjXCweNVRVcp8lZ8R0HmXwfgevlCNz/GQo=
      -----END CERTIFICATE-----
      subject=/C=CA/ST=Ontario/L=Ottawa/O=SIP Inspector Ltd/OU=Development/CN=192.168.1.99
      issuer=/C=CA/ST=Ontario/L=Ottawa/O=SIP Inspector Ltd/OU=Development/CN=192.168.1.99
      ---
      No client certificate CA names sent
      ---
      SSL handshake has read 2309 bytes and written 509 bytes
      ---
      New, TLSv1/SSLv3, Cipher is ECDHE-RSA-DES-CBC3-SHA
      Server public key is 4096 bit
      Secure Renegotiation IS supported
      Compression: NONE
      Expansion: NONE
      SSL-Session:
          Protocol  : TLSv1.2
          Cipher    : ECDHE-RSA-DES-CBC3-SHA
          Session-ID:       5755C781D91CF3177DF624EA3599EE430DAB4790F325FAD9378FEAE7731C4497
          Session-ID-ctx: 
          Master-Key: D149008E43E29D658D29418C9F770B3D6018B1D7CA2F493027B0AC7C3BA8E53B572B68C371153568B8988A1E5F351839
          Key-Arg   : None
          PSK identity: None
      PSK identity hint: None
          SRP username: None
          Start Time: 1465239425
          Timeout   : 300 (sec)
          Verify return code: 0 (ok)
       ---


Here's the command and output when I run my app that tries to connect to the same in-house tool which fails:

    > ./client3 192.168.1.99
    Enter PEM pass phrase:
    connecting to 192.168.1.99:16001
     -Error with certificate at depth: 0
       issuer   = /C=CA/ST=Ontario/L=Ottawa/O=SIP Inspector Ltd/OU=Development   /CN=192.168.1.99
       subject  = /C=CA/ST=Ontario/L=Ottawa/O=SIP Inspector Ltd/OU=Development/CN=192.168.1.99
       err 18:self signed certificate
     ** client3.c:94 Error connecting SSL object
    139788992993088:error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed:s3_clnt.c:1180:
    > 

Here are the api's I call in the my app that utilize the same credentials used by the s_client command:

 SSL_CTX_new(SSLv23_method());
 SSL_CTX_load_verify_locations(ctx, "../_security/SipInspector/certificate.pem", NULL)
 SSL_CTX_use_PrivateKey_file(ctx, "../_security/client.pem", SSL_FILETYPE_PEM)
 SSL_CTX_set_verify(ctx, SSL_VERIFY_PEER, verify_callback);
 SSL_CTX_set_verify_depth(ctx, 4);
 SSL_CTX_set_options(ctx, SSL_OP_ALL | SSL_OP_NO_SSLv2);

And also I used the openssl verify command to double check the certificate against itself (not sure if this really does anything).

Any help would be appreciated.



Carl Heyendal | Software Developer 
1826 Robertson Road | Ottawa, ON K2H 5Z6 | CANADA
Office: +1 613-725-2980 x149





***  Please note that this message and any attachments may contain confidential and proprietary material and information and are intended only for the use of the intended recipient(s). If you are not the intended recipient, you are hereby notified that any review, use, disclosure, dissemination, distribution or copying of this message and any attachments is strictly prohibited. If you have received this email in error, please immediately notify the sender and destroy this e-mail and any attachments and all copies, whether electronic or printed. Please also note that any views, opinions, conclusions or commitments expressed in this message are those of the individual sender and do not necessarily reflect the views of Fortinet, Inc., its affiliates, and emails are not binding on Fortinet and only a writing manually signed by Fortinet's General Counsel can be a binding commitment of Fortinet to Fortinet's customers or partners. Thank you. ***