Ferry Toth wrote: > Hi, > > Op 02-04-2021 om 22:16 schreef Thinh Nguyen: >> Ferry Toth wrote: >>> Hi >>> >>> Op 30-03-2021 om 23:57 schreef Ferry Toth: >>>> Hi >>>> >>>> Op 30-03-2021 om 22:26 schreef Ferry Toth: >>>>> Hi, >>>>> >>>>> Op 30-03-2021 om 18:17 schreef Felipe Balbi: >>>>>> Hi, >>>>>> >>>>>> Andy Shevchenko <andy.shevchenko@xxxxxxxxx> writes: >>>>>>> Hi! >>>>>>> >>>>>>> I have a platform with DWC3 in Dual Role mode. Currently I'm >>>>>>> experimenting on v5.12-rc5 with a few patches (mostly configuration) >>>>>>> applied [1]. I'm using Debian Unstable on the host machine and >>>>>>> BuildRoot with the above mentioned kernel on the target. >>>>>>> >>>>>>> **So, scenario 0: >>>>>>> 1. Run iperf3 -s on target >>>>>>> 2. Run iperf3 -c ... -t 0 on the host >>>>>>> 3. 0.00-10.36 sec 237 MBytes 192 Mbits/sec >>>>>>> receiver >>>>>>> >>>>>>> **Scenario 1: >>>>>>> 1. Now, detach USB cable, wait for several seconds, attach it back, >>>>>>> repeat above: >>>>>>> 0.00-9.94 sec 209 MBytes 176 Mbits/sec receiver >>>>>>> >>>>>>> Note the bandwidth drop (177 vs. 192). >>>>>>> >>>>>>> (Repeating scenario 1 will give now the same result) >>>>>>> >>>>>>> **Scenario 2. >>>>>>> 1. Detach USB cable, attach a device, for example USB stick, >>>>>>> 2. See it being enumerated and detach it. >>>>>>> 3. Attach cable from host >>>>>>> 4 . 0.00-19.36 sec 315 MBytes 136 Mbits/sec >>>>>>> receiver >>>>>>> >>>>>>> Note even more bandwidth drop! >>>>>>> >>>>>>> (Repeating scenario 1 keeps the same lower bandwidth) >>>>>>> >>>>>>> NOTE, sometimes on this scenario after several seconds the target >>>>>>> simply reboots (w/o any logs [from kernel] printed)! >>>>>>> >>>>>>> So, any pointers on how to debug and what can be a smoking gun here? >>>>>>> >>>>>>> Ferry reported this in [2]. There are different kernel versions and >>>>>>> tools to establish the connection (like connman vs. none in my >>>>>>> case). >>>>>>> >>>>>>> [1]: >>>>>>> https://urldefense.com/v3/__https://github.com/andy-shev/linux/__;!!A4F2R9G_pg!KpQnudHIK6XgK6HbPaqtbVgipDmkNBWewo-euAIuBlGdtSiaQiJ8jLn9OoMEppG6qq-d$ >>>>>>> >>>>>>> [2]: >>>>>>> https://urldefense.com/v3/__https://github.com/andy-shev/linux/issues/31__;!!A4F2R9G_pg!KpQnudHIK6XgK6HbPaqtbVgipDmkNBWewo-euAIuBlGdtSiaQiJ8jLn9OoMEptMCrp-F$ >>>>>>> >>>>>> >>>>>> dwc3 tracepoints should give some initial hints. Look at packets >>>>>> sizes >>>>>> and period of transmission. From dwc3 side, I can't think of >>>>>> anything we >>>>>> would do to throttle the transmission, but tracepoints should tell a >>>>>> clearer story. >>>>>> >>>>> My testing (but yes, with difference kernel and network managed by >>>>> connman) shows: >>>>> >>>>> 1) on cold boot eem network gadget works fine >>>>> >>>>> 2) after unplug or warm reboot (which is also an unplug) it's broken, >>>>> speed is lost (|12.0 Mbits/sec from 200Mb/s normally)|, packets lost, >>>>> no configuration received from dhcp, occasional reboot, only way to >>>>> fix is cold boot >>>>> >>>>> 3) if before unplug `connmanctl disable gadget`, on replugging and >>>>> enabling it works fine >>>>> >>>>> My theory is that some HW register is disturbed on a surprise unplug, >>>>> but not reset on plug or warm boot. But on cold boot is cleared. >>>>> Maybe that can help to narrow down tracepoints? >>>>> >>>> I captured a plug after warm and after cold boot. This includes >>>> network setup (dhcp). You can find it in [2] or directly link here: >>>> https://urldefense.com/v3/__https://github.com/andy-shev/linux/files/6232410/boot.zip__;!!A4F2R9G_pg!KpQnudHIK6XgK6HbPaqtbVgipDmkNBWewo-euAIuBlGdtSiaQiJ8jLn9OoMEpjhhrwa-$ >>>> >>> >>> >>> While the above traces in boot.zip allow compare which regs not >>> correctly initialized on warm boot, I have now captured traces of >>> unplug/plug. >>> >>> Here kernel is 5.10.27 (LTS), cold booted with USB cable plugged and the >>> eem gadget network setup (dhcp). Then trace unplug. Then trace plug. >>> >>> After plug the eem connection is again broken. >>> >>> This might allow figuring out what goes wrong on unplug. Traces here: >>> https://urldefense.com/v3/__https://github.com/andy-shev/linux/files/6250924/plug-unplug.zip__;!!A4F2R9G_pg!KpQnudHIK6XgK6HbPaqtbVgipDmkNBWewo-euAIuBlGdtSiaQiJ8jLn9OoMEpgii82NS$ >>> >>> ** >>> >> >> Hi, >> >> Were you able to narrow down the issue to only DWC3 device? (i.e. you >> tested with different hosts and different device controllers to confirm >> this) > > I haven't tried with other devices. I have been forced to replace my > host mobo and nothing changed. But I didn't pay attention to the > particular host controller. > It'd be better if we can narrow down the culprit as this seems to me like a synchronization issue at the upper layer between the host and device. >> Did you see this issue previously? If not, is it possible to do git >> bisection? > > This is with Intel Edison where main line usb gadget support appeared > around 4.19 iirc. I believed the problem appeared between 5.4 and 5.7 > and tried to bisect but failed. > > I realize only now that I failed because: > 1) 5.4 already has this issue as I recently retested I'm confused, why do you believe the problem is between 5.4 and 5.7 if 5.4 already has this issue? So when did you start seeing this problem? Also, these kernel versions are really old, there's been a lot of updates/fixes to dwc3 since then. Can we run tests on the latest kernel? > 2) I didn't use a reproducible criterion. After warm reboot the eem > gadget fails, but you can flip the host/gadget switch back and forth and > have the illusion that the connection restored. > > The scenario described here is reproducible: leaving the switch in > gadget mode eem works after cold boot only. And it likely breaks on unplug. > > A 2nd hint is that disabling gadget (I used `connmanctl disable gadget` > but I believe that has the same effect as `iw link set dev usb0 down`) > before unplug prevents messing up the driver, so you can replug and > enable again. These data points are good. However, we'd need to know where to look first. The issue isn't obvious from the DWC3 controller or the DWC3 driver. Can you check a few things: 1) Any error/timeout messages from the host's dmesg? Or device side? 2) What kernel version is your host using? Can you use the latest for both host and device? 3) Snapshot of dwc3 tracepoints of active transfers between the normal vs throttled of the latest kernel BR, Thinh
