Hello, I have found a deadlock in the n_gsm line discipline. In the function gsm_modem_upd_via_msc, it sends a gsm control message and does a blocking wait (gsm_control_wait) until either a response or timeout. The problem happens when a modem update is triggered inside the callback for incoming data "gsmld_receive_buf" gsmld_receive_buf -> gsm0_receive -> gsm_queue -> gsm_dlci_open -> gsm_modem_update -> gsm_modem_upd_via_msc after a "UA" response in gsm_queue. When line disciplines are flushed in flush_to_ldisc, there is a lock making this section of code including all of the GSM command processing single threaded. The blocking call in gsm_modem_upd_via_msc (gsm_control_wait) can only be satisfied if a new GSM response is received and processed in another callback to gsmld_receive_buf, which cannot happen because we are blocking in that locked function. The code eventually does recover after all of the T1 timeouts and retries. At that point, all the responses which were queued up but could not be processed come in, but the state machine had already closed the connection and is no longer waiting for the response. I believe this is the only situation that causes the deadlock because all other calls to gsm_modem_upd_via_msc are initiated through a different tty_ldisc_ops callback. This deadlock only happens when encoding==GSM_BASIC_OPT, adaption == 1, and initiator == 1. This call was added in 2022 with commit 4847380. I believe to satisfy: "5.4.6.3.7 Modem Status Command (MSC)" "This command shall be sent prior to any user data after a creation of a DLC" of "3GPP TS 07.10". The code doesn't actually do anything with the response, since the response is just an echo of the data we sent, so we could simply just not wait for the response in this case. Or we could make it a new opening state and process the response async similar to how CMD_PN is done currently.
