I checked this further and the issue was indeed with my code.. I did some recent async io completion handler refactor because of which the bio's socket write completion was triggering the observer's read completion callback.. the records were actually client side write bio buffers which got mixed with the ssl_read data on the read path. Thanks for the pointer, we are doing good. Nothing further is needed at this point.
Neelabh
On Thu, Nov 24, 2022 at 4:51 PM Neelabh Mam <neelabh.mam@xxxxxxxxx> wrote:
I hook an observer for decrypted data immediately after the handshake is successful (SSL_do_handshake rc 1) and it is this observer which gets the ccs+list data on the vert next ssl_read cycle. Now, it could be that my code is at fault here.. But I do see the decrypted dummy ccs and one more record as part of the list data. Annotated hex dump of the buffer reproduced below. Btw this is against the latest FZ windows FTPS server which I think recently moved to GNUtls. As suggested earlier I'll check my code again and open a github ticket.Command : PASV
sock-cc 2570925053120 OnWrite, n 29
sock-cc 2570925053120 OnRead, n 273
New session callback 1
sock-cc 2570925053120 OnRead, n 68
Response : 227 Entering Passive Mode (127,0,0,1,249,75)
Command : LIST /
sock-dc 2570925624800 OnConnect, n 0
sock-dc OnConnect()
sock-cc 2570925053120 OnWrite, n 30
sock-cc 2570925053120 OnRead, n 51
Response : 150 Starting data transfer.
sock-dc 2570925624800 OnWrite, n 32
sock-dc 2570925624800 OnWrite, n 32
sock-dc 2570925624800 OnWrite, n 32
sock-dc 2570925624800 OnWrite, n 32
sock-dc 2570925624800 OnWrite, n 32
sock-dc 2570925624800 OnWrite, n 32
sock-dc 2570925624800 OnWrite, n 32
sock-dc 2570925624800 OnWrite, n 32
sock-dc 2570925624800 OnWrite, n 32
sock-dc 2570925624800 OnWrite, n 32
sock-dc 2570925624800 OnWrite, n 32
sock-dc 2570925624800 OnWrite, n 32
sock-dc 2570925624800 OnWrite, n 32
sock-dc 2570925624800 OnWrite, n 32
sock-dc 2570925624800 OnWrite, n 32
sock-dc 2570925624800 OnWrite, n 32
sock-dc 2570925624800 OnWrite, n 32
sock-dc 2570925624800 OnWrite, n 32
sock-dc 2570925624800 OnWrite, n 10
sock-dc 2570925624800 OnRead, n 133
sock-dc Handshake failed : -1 2
sock-dc 2570925624800 OnRead, n 108
sock-dc Handshake done. TLS session reused : 1
sock-dc 2570925624800 OnWrite, n 32
observer OnDataChannelIoCompletion write
sock-dc 2570925624800 OnWrite, n 32
observer OnDataChannelIoCompletion write
sock-dc 2570925624800 OnWrite, n 16
observer OnDataChannelIoCompletion write
sock-dc 2570925624800 OnRead, n 86
observer OnDataChannelIoCompletion read
sock-cc 2570925053120 OnRead, n 48
Response : 226 Operation successful
sock-dc 2570925624800 OnRead, n 816
sock-dc SSL_RECEIVED_SHUTDOWN, flags : 2
sock-dc : ssl_shutdown() rc : 1
sock-dc SSL_RECEIVED_SHUTDOWN, flags : 3
sock-dc : shutdown(sd_send)
observer OnDataChannelIoCompletion read
sock-dc 2570925624800 OnRead, n 0
sock-dc shutdown mode : 3
¢â╣ª5d²┼l⌐\╒═£tdrwxrwxrwx 1 ftp ftp 0 Nov 01 08:55 $RECYCLE.BIN
-rw-rw-rw- 1 ftp ftp 1468320 Oct 15 17:37 accesschk.exe
-rw-rw-rw- 1 ftp ftp 810416 Oct 15 17:37 accesschk64.exe
-rw-rw-rw- 1 ftp ftp 264088 Oct 15 17:37 AccessEnum.exe
-rw-rw-rw- 1 ftp ftp 2502032 Oct 15 17:37 Autoruns.exe
-rw-rw-rw- 1 ftp ftp 2928520 Oct 15 17:37 Autoruns64.exe
-rw-rw-rw- 1 ftp ftp 712080 Oct 15 17:37 autorunsc.exe
-rw-rw-rw- 1 ftp ftp 788400 Oct 15 17:37 autorunsc64.exe
drwxrwxrwx 1 ftp ftp 0 Nov 06 16:52 System Volume Information
-rw-rw-rw- 1 ftp ftp 5 Nov 24 02:26 x.txt14 TLS Change Cipher Spec protocol 03 03 SSL 3.3 (TLS 1.2) 00 01 length of change cipher spec message 01 payload (irrelevant)14 03 03 00 01 01 17 03 03 00 45 4b 60 8c 72 be17 TLS application data 03 03 version 1.2 00 45 length of payload (69 bytes)
99 60 ed da 3a a0 09 9c 35 24 a5 3d 48 b5 78 bc
2e 2c c3 e7 19 b4 74 e9 7b 70 90 d4 17 d4 2a 46
8c e7 1a 94 6b c8 ea 54 b2 72 8b 03 8e cb 0d 9b
83 b9 a6 35 64 fd c5 55 08 6c a9 5c d5 cd 9c 74
64 72 77 78 72 77 78 72 77 78 20 31 20 66 74 70
20 66 74 70 20 20 20 20 20 20 20 20 20 20 20 20
20 20 20 30 20 4e 6f 76 20 30 31 20 30 38 3a 35
35 20 24 52 45 43 59 43 4c 45 2e 42 49 4e 0d 0a
2d 72 77 2d 72 77 2d 72 77 2d 20 31 20 66 74 70
20 66 74 70 20 20 20 20 20 20 20 20 20 31 34 36
38 33 32 30 20 4f 63 74 20 31 35 20 31 37 3a 33
37 20 61 63 63 65 73 73 63 68 6b 2e 65 78 65 0d
0a 2d 72 77 2d 72 77 2d 72 77 2d 20 31 20 66 74
70 20 66 74 70 20 20 20 20 20 20 20 20 20 20 38
31 30 34 31 36 20 4f 63 74 20 31 35 20 31 37 3a
33 37 20 61 63 63 65 73 73 63 68 6b 36 34 2e 65:: goes on full listOn Thu, Nov 24, 2022 at 3:15 PM Matt Caswell <matt@xxxxxxxxxxx> wrote:
On 24/11/2022 07:57, Neelabh Mam wrote:
> Hi,
>
> With my openssl based FTPS client (non-blocking bio) targeting TLS1.3, I
> see that immediately after a successful data channel handshake (with
> session reuse), a dummy change_cipher_spec record and a non-application
> data record are sent as part of the directory listing data. Same holds
> true for file downloads as well..(again with session reuse). This seems
> to be in line with the last paragraph in Appendix D.4 of RFC8446 which
> mandates a change_cipher_spec record to be sent from the server in case
> of session reuse. I could manually filter out the non-application data
> records from the actual data.. However, I was wondering if there's some
> api/setting in openssl that would do that at an API level.. as this
> looks like it should ideally be part of openssl workflow. But then what
> I am also not sure about is why the receipt of new session ticket data
> on the control channel does not bubble up as application data ? Openssl,
> from what I can tell, seems to be "consuming" it (new session data)
> internally. The earlier 2 messages however, do get exposed out of
> ssl_read and eventually become part of application data.
The only data you should be getting out of an `SSL_read` call is
application data. If you're getting handshake or CCS messages as part of
app data then something has gone very surprisingly wrong. I cannot
imagine what would cause that.
You might want to open a github issue about that to dive into it in more
detail.
Matt
> Is it like :
> the message type warrants one to be exposed (change_cipher_spec) and the
> other to be handled internally (new session data) ? Could we please
> advise on openssl's standard operating workflow here ? Also, would I
> have to add logic to manually separate these 2 non-application records
> from the application data ? Is that how it is supposed to be ? Thank you.
>
> Neelabh
