Hi,
Op 03-04-2021 om 13:25 schreef Ferry Toth:
Hi,
Op 03-04-2021 om 04:02 schreef Thinh Nguyen:
Ferry Toth wrote:
Hi,
Op 02-04-2021 om 22:16 schreef Thinh Nguyen:
Ferry Toth wrote:
Hi
Op 30-03-2021 om 23:57 schreef Ferry Toth:
Hi
Op 30-03-2021 om 22:26 schreef Ferry Toth:
Hi,
Op 30-03-2021 om 18:17 schreef Felipe Balbi:
Hi,
Andy Shevchenko <andy.shevchenko@xxxxxxxxx> writes:
Hi!
I have a platform with DWC3 in Dual Role mode. Currently I'm
experimenting on v5.12-rc5 with a few patches (mostly
configuration)
applied [1]. I'm using Debian Unstable on the host machine and
BuildRoot with the above mentioned kernel on the target.
**So, scenario 0:
1. Run iperf3 -s on target
2. Run iperf3 -c ... -t 0 on the host
3. 0.00-10.36 sec 237 MBytes 192 Mbits/sec
receiver
**Scenario 1:
1. Now, detach USB cable, wait for several seconds, attach it
back,
repeat above:
0.00-9.94 sec 209 MBytes 176 Mbits/sec receiver
Note the bandwidth drop (177 vs. 192).
(Repeating scenario 1 will give now the same result)
**Scenario 2.
1. Detach USB cable, attach a device, for example USB stick,
2. See it being enumerated and detach it.
3. Attach cable from host
4 . 0.00-19.36 sec 315 MBytes 136 Mbits/sec
receiver
Note even more bandwidth drop!
(Repeating scenario 1 keeps the same lower bandwidth)
NOTE, sometimes on this scenario after several seconds the target
simply reboots (w/o any logs [from kernel] printed)!
So, any pointers on how to debug and what can be a smoking gun
here?
Ferry reported this in [2]. There are different kernel versions
and
tools to establish the connection (like connman vs. none in my
case).
[1]:
https://urldefense.com/v3/__https://github.com/andy-shev/linux/__;!!A4F2R9G_pg!KpQnudHIK6XgK6HbPaqtbVgipDmkNBWewo-euAIuBlGdtSiaQiJ8jLn9OoMEppG6qq-d$
[2]:
https://urldefense.com/v3/__https://github.com/andy-shev/linux/issues/31__;!!A4F2R9G_pg!KpQnudHIK6XgK6HbPaqtbVgipDmkNBWewo-euAIuBlGdtSiaQiJ8jLn9OoMEptMCrp-F$
dwc3 tracepoints should give some initial hints. Look at packets
sizes
and period of transmission. From dwc3 side, I can't think of
anything we
would do to throttle the transmission, but tracepoints should
tell a
clearer story.
My testing (but yes, with difference kernel and network managed by
connman) shows:
1) on cold boot eem network gadget works fine
2) after unplug or warm reboot (which is also an unplug) it's
broken,
speed is lost (|12.0 Mbits/sec from 200Mb/s normally)|, packets
lost,
no configuration received from dhcp, occasional reboot, only way to
fix is cold boot
3) if before unplug `connmanctl disable gadget`, on replugging and
enabling it works fine
My theory is that some HW register is disturbed on a surprise
unplug,
but not reset on plug or warm boot. But on cold boot is cleared.
Maybe that can help to narrow down tracepoints?
I captured a plug after warm and after cold boot. This includes
network setup (dhcp). You can find it in [2] or directly link here:
https://urldefense.com/v3/__https://github.com/andy-shev/linux/files/6232410/boot.zip
While the above traces in boot.zip allow compare which regs not
correctly initialized on warm boot, I have now captured traces of
unplug/plug.
Here kernel is 5.10.27 (LTS), cold booted with USB cable plugged
and the
eem gadget network setup (dhcp). Then trace unplug. Then trace plug.
After plug the eem connection is again broken.
This might allow figuring out what goes wrong on unplug. Traces here:
https://urldefense.com/v3/__https://github.com/andy-shev/linux/files/6250924/plug-unplug.zip
**
Hi,
Were you able to narrow down the issue to only DWC3 device? (i.e. you
tested with different hosts and different device controllers to confirm
this)
I haven't tried with other devices. I have been forced to replace my
host mobo and nothing changed. But I didn't pay attention to the
particular host controller.
It'd be better if we can narrow down the culprit as this seems to me
like a synchronization issue at the upper layer between the host and
device.
Did you see this issue previously? If not, is it possible to do git
bisection?
This is with Intel Edison where main line usb gadget support appeared
around 4.19 iirc. I believed the problem appeared between 5.4 and 5.7
and tried to bisect but failed.
I realize only now that I failed because:
1) 5.4 already has this issue as I recently retested
I'm confused, why do you believe the problem is between 5.4 and 5.7 if
5.4 already has this issue? So when did you start seeing this problem?
Because at the time of 5.4 I didn't notice the issue as I normally did
cold boots due to other problems on warm boot (i.e. sdhc inaccessible).
I never new that on a cold boot it works. Even during bisecting I didn't
know until the end, and then I found 5.4 has the same problem as all the
later kernels (tested up to 5.11)
Also, these kernel versions are really old, there's been a lot of
updates/fixes to dwc3 since then. Can we run tests on the latest kernel?
I have tested 5.10.27, 5.11.0 and 5.11.4-rt11.
But of course I am completely prepared to run Andy's latest (v5.12-rc5)
on the device.
2) I didn't use a reproducible criterion. After warm reboot the eem
gadget fails, but you can flip the host/gadget switch back and forth and
have the illusion that the connection restored.
The scenario described here is reproducible: leaving the switch in
gadget mode eem works after cold boot only. And it likely breaks on
unplug.
A 2nd hint is that disabling gadget (I used `connmanctl disable gadget`
but I believe that has the same effect as `iw link set dev usb0 down`)
before unplug prevents messing up the driver, so you can replug and
enable again.
These data points are good. However, we'd need to know where to look
first. The issue isn't obvious from the DWC3 controller or the DWC3
driver.
Can you check a few things:
1) Any error/timeout messages from the host's dmesg? Or device side?
I'll add log from the host side.
For now I only see (on a warm plug):
kernel: usb 1-11: can't set config #1, error -110
2) What kernel version is your host using? Can you use the latest for
both host and device?
The host is ubuntu's amd64 5.8.0-48-generic.
I will test with v5.12-rc5 from ubuntu kernel ppa on the host. And
Andy's latest (v5.12-rc5) on the device.
I upgraded host kernel, but not yet device and captured relevant host
journal messages and device traces. Something did change: after cold
boot I don't a eem until after I unplug/replug. I then traced a iperf
transfer. Then after again unplug/replug I get the throttled connection,
which I also traced.
See https://github.com/andy-shev/linux/files/6253414/transfer.zip
I am expecting results this evening.
3) Snapshot of dwc3 tracepoints of active transfers between the normal
vs throttled of the latest kernel
I don't know if the problem I see is really throttling.
I can trace an active transfer, but that does actually throttle from
200Mb/s down to 139MB/s and produces a trace of 53MB. (2x1sec of iperf3).
BR,
Thinh
