Re: Openssl seems to inspects application data?

["Kreissl, Jochen" <Jochen.Kreissl@xxxxxxxxxx>]

I am getting it on ssl_read.

 

Still debugging. Right now, it seems that our custom BIO is called three times.

- First ssl reads 5 bytes (header).

- Second: we fetch some 1600-ish bytes. Still not enough for the entire record (the chain is roughly 7 kb long)

- Last: our BIO is running out of data (network packages not available yet) and returns a 0 and sets the BIO flag to retry_send.  We have this retry_send behavior in other places too and it works (e.g. during handshake with the certificate message).

 

But in this instance, somewhere in the internal of ssl_read, an error occurs following the return code 0.

We get a -1 from ssl_read and then call SSL_get_error – which gives us a fatal SSL_ERROR_SSL.

We then call ERR_get_error and get the beforementioned, weird error code.

I’m still trying to find the exact spot where the internals of ssl_read fail.

 

---

From: Neil Horman <nhorman@xxxxxxxxxxx>
Sent: Tuesday, March 26, 2024 6:55:25 PM
To: Kreissl, Jochen <Jochen.Kreissl@xxxxxxxxxx>
Cc: openssl-users@xxxxxxxxxxx <openssl-users@xxxxxxxxxxx>
Subject: Re: Openssl seems to inspects application data?

 

Sie erhalten nicht oft eine E-Mail von nhorman@xxxxxxxxxxx. Erfahren Sie, warum dies wichtig ist

What library call are you getting that error in response to?  If you believe that this is coming from some attempt to interpret application data (which you are correct, it shouldn't be, unless the application auth protocol is somehow getting aliased as a tls control message of some sort), then I would, after the handshake, clear the error stack, and check it after a call from SSL_read returns.

 

On Tue, Mar 26, 2024 at 1:38 PM Kreissl, Jochen <Jochen.Kreissl@xxxxxxxxxx> wrote:

Hi,

 

I am using openssl (3.2) in an application.

Handshake works just fine but I get a very weird behavior when I receive a big certificate chain inside application data (TLS 1.3 but NOT using Post-Handshake Auth, this is some level-7 auth protocol on top of tls).

The openssl error I get is error:0308010C:digital envelope routines::unsupported

Which … seems to indicate that openssl is trying (and failing) to interpret the certificate chain…?

 

I really don’t understand what is going on.

I thought openssl would treat any application data sent using SSL_write following a completed handshake would be opaque for openssl – because why would it look inside and try to parse something?

 

Does anyone have an explanation or have encountered something similar?

 

 

Regards

Jochen