Hello!
On 05/20/2019 10:42 AM, Yoshihiro Shimoda wrote:
> Thank you for the reply! And, I'm sorry for the delayed response because I had days off in last week.
And sorry for my delayed reply as well!
>> It's not "some SoCs", it's only R-Car gen2 and RZ/G1 SoCs.
>
> Do you prefer the subject "net: sh_eth: fix mdio access in sh_eth_close() for R-Car gen2 and RZ/G1 SoCs"
> instead of "... some SoCs"?
Yes, it's just about a single SoC family as the driver knows them, really: R-Car gen2;
RZ/G1 uses the same SoC data.
>>> The sh_eth_close() resets the MAC and then calls phy_stop()
>>> so that mdio read access result is incorrect without any error
>>> according to kernel trace like below:
>>>
>>> ifconfig-216 [003] .n.. 109.133124: mdio_access: ee700000.ethernet-ffffffff read phy:0x01 reg:0x00 val:0xffff
>>
>> Not sure how RMII mode affects the MDIO transfers...
>
> I should have described what happen on the previous code.
> According to the hardware manual, the RMII mode should be set to 1 before
> operation the Ethernet MAC. The MDIO transferring is one of the Ethernet MAC operation.
> So, I assumed this RMII mode affects the MDIO transfers.
>
> By the way, I also should have described what happen on the previous code.
> If I input the following command on non-NFS rootfs environment (e.g. USB memory rootfs),
> sometimes, a lot of FCD interruption happened and then the following "rcu "error happened.
> ---
> # ifconfig eth0 down# ifconfig eth0 up
> [ 794.545823] Micrel KSZ8041RNLI ee700000.ethernet-ffffffff:01: attached PHY driver [Micrel KSZ8041RNLI] (mii_bus:phy_addr=ee700000.ethernet-ffffffff:01, irq=157)
> # ifconfig eth0 up# ifconfig eth0 down
> # ifconfig eth0 down# ifconfig eth0 up
> [ 797.662138] Micrel KSZ8041RNLI ee700000.ethernet-ffffffff:01: attached PHY driver [Micrel KSZ8041RNLI] (mii_bus:phy_addr=ee700000.ethernet-ffffffff:01, irq=157)
> [ 797.689158] sh-eth ee700000.ethernet eth0: Link is Up - 100Mbps/Full - flow control rx/tx
> [ 797.702631] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
> # [ 818.759628] rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
> [ 818.770827] rcu: 0-....: (2 GPs behind) idle=d12/0/0x3 softirq=388/388 fqs=1051
> [ 818.783635] rcu: (detected by 1, t=2104 jiffies, g=377, q=18)
> [ 818.794848] Sending NMI from CPU 1 to CPUs 0:
> [ 818.804733] NMI backtrace for cpu 0
> [ 818.804736] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.1.0-00002-g192664e-dirty #27
> [ 818.804738] Hardware name: Generic R-Car Gen2 (Flattened Device Tree)
> [ 818.804740] PC is at __do_softirq+0x7c/0x344
> [ 818.804742] LR is at irq_exit+0x6c/0xcc
> [ 818.804744] pc : [<c01021e4>] lr : [<c0126aa0>] psr: 60030113
> [ 818.804746] sp : c0d01eb8 ip : 00200102 fp : c0d03080
> [ 818.804748] r10: 00000001 r9 : 0000000a r8 : c0ca2940
> [ 818.804750] r7 : 00000004 r6 : c0ca2934 r5 : 00000000 r4 : ffffe000
> [ 818.804752] r3 : c0ca2940 r2 : 2aaa8000 r1 : 00000000 r0 : 2aaa8000
> [ 818.804754] Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment none
> [ 818.804756] Control: 10c5387d Table: 6a5f006a DAC: 00000051
> [ 818.804758] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.1.0-00002-g192664e-dirty #27
> [ 818.804760] Hardware name: Generic R-Car Gen2 (Flattened Device Tree)
> [ 818.804762] [<c011049c>] (unwind_backtrace) from [<c010bc0c>] (show_stack+0x18/0x1c)
> [ 818.804764] [<c010bc0c>] (show_stack) from [<c07f7458>] (dump_stack+0x7c/0x9c)
> [ 818.804766] [<c07f7458>] (dump_stack) from [<c07fd358>] (nmi_cpu_backtrace+0x98/0xb8)
> [ 818.804768] [<c07fd358>] (nmi_cpu_backtrace) from [<c010e6bc>] (handle_IPI+0x2bc/0x33c)
> [ 818.804770] [<c010e6bc>] (handle_IPI) from [<c03fa084>] (gic_handle_irq+0x8c/0x98)
> [ 818.804773] [<c03fa084>] (gic_handle_irq) from [<c0101a8c>] (__irq_svc+0x6c/0x90)
> [ 818.804774] Exception stack(0xc0d01e68 to 0xc0d01eb0)
> [ 818.804777] 1e60: 2aaa8000 00000000 2aaa8000 c0ca2940 ffffe000 00000000
> [ 818.804779] 1e80: c0ca2934 00000004 c0ca2940 0000000a 00000001 c0d03080 00200102 c0d01eb8
> [ 818.804781] 1ea0: c0126aa0 c01021e4 60030113 ffffffff
> [ 818.804783] [<c0101a8c>] (__irq_svc) from [<c01021e4>] (__do_softirq+0x7c/0x344)
> [ 818.804785] [<c01021e4>] (__do_softirq) from [<c0126aa0>] (irq_exit+0x6c/0xcc)
> [ 818.804787] [<c0126aa0>] (irq_exit) from [<c016c344>] (__handle_domain_irq+0x88/0xbc)
> [ 818.804789] [<c016c344>] (__handle_domain_irq) from [<c03fa058>] (gic_handle_irq+0x60/0x98)
> [ 818.804791] [<c03fa058>] (gic_handle_irq) from [<c0101a8c>] (__irq_svc+0x6c/0x90)
> [ 818.804793] Exception stack(0xc0d01f38 to 0xc0d01f80)
> [ 818.804795] 1f20: 00000000 0008bbb8
> [ 818.804798] 1f40: eb74b520 c011aee0 00000001 ffffe000 c0d08c30 c0d8e5be c0d08c74 c0d980c0
> [ 818.804800] 1f60: 00000001 c0c5ea40 00000000 c0d01f88 c01084ec c014c85c 60030013 ffffffff
> [ 818.804802] [<c0101a8c>] (__irq_svc) from [<c014c85c>] (do_idle+0xf0/0x13c)
> [ 818.804804] [<c014c85c>] (do_idle) from [<c014cb10>] (cpu_startup_entry+0x20/0x24)
> [ 818.804806] [<c014cb10>] (cpu_startup_entry) from [<c0c00fac>] (start_kernel+0x3e0/0x488)
This *usually* means a register is accessed while the clock is off...
> Note:
> - After I input the second ifconfig eth0 up, I cannot input any command.
> - If I disconnect an ethernet cable, I can input command again.
Hm...
> ---
>
>>> To fix the issue, this patch adds a condition and set the RMII mode
>>> regiseter in sh_eth_dev_exit() for some SoCs.
Register.
>>> Note that when I have tried to move the sh_eth_dev_exit() calling
>>> after phy_stop() on sh_eth_close(), but it gets worse.
>>
>> Ah, I was going to suggest changing the call order... what happens then?
>
> Sometimes, the following error happened and then I cannot recover the system
> even if I disconnected an ethernet cable. Do you prefer this way, instead of
> adding RMII mode setting on sh_eth_dev_exit()?
> ---
> # ifconfig eth0 up;
> [ 64.896021] Micrel KSZ8041RNLI ee700000.ethernet-ffffffff:01: attached PHY driver [Micrel KSZ8041RNLI] (mii_bus:phy_addr=ee700000.ethernet-ffffffff:01, irq=157)
> # [ 66.868241] sh-eth ee700000.ethernet eth0: Link is Up - 100Mbps/Full - flow control rx/tx
> [ 66.881777] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
> # ifconfig eth0 down
> [ 71.164900] sh-eth ee700000.ethernet eth0: Link is Down
> [ 92.169767] rcu: INFO: rcu_sched self-detected stall on CPU
> [ 92.180654] rcu: 0-....: (2100 ticks this GP) idle=9d6/1/0x40000004 softirq=513/513 fqs=1050
> [ 92.194720] rcu: (t=2102 jiffies g=-367 q=46)
> [ 92.204723] NMI backtrace for cpu 0
> [ 92.213705] CPU: 0 PID: 228 Comm: ifconfig Not tainted 5.1.0-00002-g192664e-dirty #28
> [ 92.227149] Hardware name: Generic R-Car Gen2 (Flattened Device Tree)
> [ 92.239258] [<c011049c>] (unwind_backtrace) from [<c010bc0c>] (show_stack+0x18/0x1c)
> [ 92.252683] [<c010bc0c>] (show_stack) from [<c07f7458>] (dump_stack+0x7c/0x9c)
> [ 92.265548] [<c07f7458>] (dump_stack) from [<c07fd360>] (nmi_cpu_backtrace+0xa0/0xb8)
> [ 92.279039] [<c07fd360>] (nmi_cpu_backtrace) from [<c07fd3fc>] (nmi_trigger_cpumask_backtrace+0x84/0x114)
> [ 92.294384] [<c07fd3fc>] (nmi_trigger_cpumask_backtrace) from [<c01827e4>] (rcu_dump_cpu_stacks+0xb4/0xd0)
> [ 92.309893] [<c01827e4>] (rcu_dump_cpu_stacks) from [<c0180cc8>] (rcu_sched_clock_irq+0x2cc/0x6f0)
> [ 92.324744] [<c0180cc8>] (rcu_sched_clock_irq) from [<c0189170>] (update_process_times+0x38/0x64)
> [ 92.339528] [<c0189170>] (update_process_times) from [<c019a5d0>] (tick_nohz_handler+0xd4/0x128)
> [ 92.354261] [<c019a5d0>] (tick_nohz_handler) from [<c06573a4>] (arch_timer_handler_virt+0x30/0x38)
> [ 92.369207] [<c06573a4>] (arch_timer_handler_virt) from [<c0171548>] (handle_percpu_devid_irq+0xf0/0x224)
> [ 92.384817] [<c0171548>] (handle_percpu_devid_irq) from [<c016c2ac>] (generic_handle_irq+0x20/0x30)
> [ 92.399910] [<c016c2ac>] (generic_handle_irq) from [<c016c364>] (__handle_domain_irq+0xa8/0xbc)
> [ 92.414678] [<c016c364>] (__handle_domain_irq) from [<c03fa058>] (gic_handle_irq+0x60/0x98)
> [ 92.429128] [<c03fa058>] (gic_handle_irq) from [<c0101a8c>] (__irq_svc+0x6c/0x90)
> [ 92.442748] Exception stack(0xea1f9b50 to 0xea1f9b98)
> [ 92.453924] 9b40: 2aaa8000 00000000 2aaa8000 c0ca2940
> [ 92.468345] 9b60: ffffe000 00000000 c0ca2934 00000002 c0ca2940 0000000a 00000000 c0d03080
> [ 92.482764] 9b80: 00404100 ea1f9ba0 c0126aa0 c01021e4 60030113 ffffffff
> [ 92.495519] [<c0101a8c>] (__irq_svc) from [<c01021e4>] (__do_softirq+0x7c/0x344)
> [ 92.508973] [<c01021e4>] (__do_softirq) from [<c0126aa0>] (irq_exit+0x6c/0xcc)
> [ 92.522215] [<c0126aa0>] (irq_exit) from [<c016c344>] (__handle_domain_irq+0x88/0xbc)
> [ 92.536092] [<c016c344>] (__handle_domain_irq) from [<c03fa058>] (gic_handle_irq+0x60/0x98)
> [ 92.550513] [<c03fa058>] (gic_handle_irq) from [<c0101a8c>] (__irq_svc+0x6c/0x90)
> [ 92.564095] Exception stack(0xea1f9c20 to 0xea1f9c68)
> [ 92.575129] 9c20: f0911320 00000001 00000000 f0911320 00000003 eabb7844 00000001 00000000
> [ 92.589271] 9c40: 00000000 00003800 00000000 ea1f9e50 ea1f9d24 ea1f9c70 c0555d80 c055c590
> [ 92.603319] 9c60: 60030013 ffffffff
> [ 92.612529] [<c0101a8c>] (__irq_svc) from [<c055c590>] (sh_mdio_ctrl+0x3c/0x58)
> [ 92.625531] [<c055c590>] (sh_mdio_ctrl) from [<c0555d80>] (mdiobb_send_num+0x28/0x34)
> [ 92.638971] [<c0555d80>] (mdiobb_send_num) from [<c0555f10>] (mdiobb_write+0x70/0x94)
> [ 92.652337] [<c0555f10>] (mdiobb_write) from [<c05555a4>] (__mdiobus_write+0x64/0x10c)
> [ 92.665688] [<c05555a4>] (__mdiobus_write) from [<c05526e8>] (__phy_modify_changed+0x58/0x6c)
> [ 92.679587] [<c05526e8>] (__phy_modify_changed) from [<c0552760>] (__phy_modify+0x10/0x18)
> [ 92.693127] [<c0552760>] (__phy_modify) from [<c05527a4>] (phy_modify+0x3c/0x54)
> [ 92.705749] [<c05527a4>] (phy_modify) from [<c0553b14>] (phy_suspend+0xb0/0xd0)
> [ 92.718274] [<c0553b14>] (phy_suspend) from [<c0551118>] (phy_state_machine+0x1a4/0x1a8)
> [ 92.731589] [<c0551118>] (phy_state_machine) from [<c05511a8>] (phy_stop+0x8c/0xb8)
> [ 92.744411] [<c05511a8>] (phy_stop) from [<c055e2d4>] (sh_eth_close+0x58/0xa0)
> [ 92.756668] [<c055e2d4>] (sh_eth_close) from [<c06ce740>] (__dev_close_many+0xc0/0xdc)
> [ 92.769620] [<c06ce740>] (__dev_close_many) from [<c06d4f9c>] (__dev_change_flags+0xdc/0x1b0)
> [ 92.783186] [<c06d4f9c>] (__dev_change_flags) from [<c06d5090>] (dev_change_flags+0x20/0x50)
> [ 92.796660] [<c06d5090>] (dev_change_flags) from [<c074d740>] (devinet_ioctl+0x2d8/0x5dc)
> [ 92.809892] [<c074d740>] (devinet_ioctl) from [<c074f8f8>] (inet_ioctl+0x260/0x324)
> [ 92.822560] [<c074f8f8>] (inet_ioctl) from [<c06b2ea4>] (sock_ioctl+0x148/0x424)
> [ 92.834968] [<c06b2ea4>] (sock_ioctl) from [<c0252e00>] (vfs_ioctl+0x28/0x3c)
> [ 92.847110] [<c0252e00>] (vfs_ioctl) from [<c0253614>] (do_vfs_ioctl+0x98/0x844)
> [ 92.859518] [<c0253614>] (do_vfs_ioctl) from [<c0253e00>] (ksys_ioctl+0x40/0x5c)
> [ 92.871875] [<c0253e00>] (ksys_ioctl) from [<c0101000>] (ret_fast_syscall+0x0/0x54)
> [ 92.884498] Exception stack(0xea1f9fa8 to 0xea1f9ff0)
> [ 92.894477] 9fa0: 000ab853 bec4fc90 00000003 00008914 bec4fc90 000ab853
> [ 92.907648] 9fc0: 000ab853 bec4fc90 bec4ff52 00000036 bec4fe48 000000b8 00096710 00000000
> [ 92.920805] 9fe0: 000ca1e4 bec4fc2c 0001aa8c b6e72f3c
> ---
Again, usually means register access while the module clock is off...
>>> Signed-off-by: Yoshihiro Shimoda <yoshihiro.shimoda.uh@xxxxxxxxxxx>
>>> ---
>>> drivers/net/ethernet/renesas/sh_eth.c | 4 ++++
>>> 1 file changed, 4 insertions(+)
>>>
>>> diff --git a/drivers/net/ethernet/renesas/sh_eth.c b/drivers/net/ethernet/renesas/sh_eth.c
>>> index e33af37..106ae90 100644
>>> --- a/drivers/net/ethernet/renesas/sh_eth.c
>>> +++ b/drivers/net/ethernet/renesas/sh_eth.c
>>> @@ -1596,6 +1596,10 @@ static void sh_eth_dev_exit(struct net_device *ndev)
>>>
>>> /* Set MAC address again */
>>> update_mac_address(ndev);
>>> +
>>> + /* Set the mode again if required */
>>
>> Should be "RMII mode", n ot just "Mode". We prolly need more detailed explanation...
>
> I got it.
Your fix seems legit now, just misplaced a bit...
> Best regards,
> Yoshihiro Shimoda
>
>>> + if (mdp->cd->rmiimode)
>>> + sh_eth_write(ndev, 0x1, RMIIMODE);
According to the gen2 manual, this register should be written right after the soft reset
via EDMR.
>>> }
>>>
>>> static void sh_eth_rx_csum(struct sk_buff *skb)
MBR, Sergei
