Hello Sergei-san, Thank you for the reply! And, I'm sorry for the delayed response because I had days off in last week. > From: Sergei Shtylyov, Sent: Thursday, May 16, 2019 12:41 AM > > Hello! > > It's not "some SoCs", it's only R-Car gen2 and RZ/G1 SoCs. Do you prefer the subject "net: sh_eth: fix mdio access in sh_eth_close() for R-Car gen2 and RZ/G1 SoCs" instead of "... some SoCs"? > On 05/15/2019 08:36 AM, Yoshihiro Shimoda wrote: > > > The sh_eth_close() resets the MAC and then calls phy_stop() > > so that mdio read access result is incorrect without any error > > according to kernel trace like below: > > > > ifconfig-216 [003] .n.. 109.133124: mdio_access: ee700000.ethernet-ffffffff read phy:0x01 reg:0x00 val:0xffff > > Not sure how RMII mode affects the MDIO transfers... I should have described what happen on the previous code. According to the hardware manual, the RMII mode should be set to 1 before operation the Ethernet MAC. The MDIO transferring is one of the Ethernet MAC operation. So, I assumed this RMII mode affects the MDIO transfers. By the way, I also should have described what happen on the previous code. If I input the following command on non-NFS rootfs environment (e.g. USB memory rootfs), sometimes, a lot of FCD interruption happened and then the following "rcu "error happened. --- # ifconfig eth0 down# ifconfig eth0 up [ 794.545823] Micrel KSZ8041RNLI ee700000.ethernet-ffffffff:01: attached PHY driver [Micrel KSZ8041RNLI] (mii_bus:phy_addr=ee700000.ethernet-ffffffff:01, irq=157) # ifconfig eth0 up# ifconfig eth0 down # ifconfig eth0 down# ifconfig eth0 up [ 797.662138] Micrel KSZ8041RNLI ee700000.ethernet-ffffffff:01: attached PHY driver [Micrel KSZ8041RNLI] (mii_bus:phy_addr=ee700000.ethernet-ffffffff:01, irq=157) [ 797.689158] sh-eth ee700000.ethernet eth0: Link is Up - 100Mbps/Full - flow control rx/tx [ 797.702631] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready # [ 818.759628] rcu: INFO: rcu_sched detected stalls on CPUs/tasks: [ 818.770827] rcu: 0-....: (2 GPs behind) idle=d12/0/0x3 softirq=388/388 fqs=1051 [ 818.783635] rcu: (detected by 1, t=2104 jiffies, g=377, q=18) [ 818.794848] Sending NMI from CPU 1 to CPUs 0: [ 818.804733] NMI backtrace for cpu 0 [ 818.804736] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.1.0-00002-g192664e-dirty #27 [ 818.804738] Hardware name: Generic R-Car Gen2 (Flattened Device Tree) [ 818.804740] PC is at __do_softirq+0x7c/0x344 [ 818.804742] LR is at irq_exit+0x6c/0xcc [ 818.804744] pc : [<c01021e4>] lr : [<c0126aa0>] psr: 60030113 [ 818.804746] sp : c0d01eb8 ip : 00200102 fp : c0d03080 [ 818.804748] r10: 00000001 r9 : 0000000a r8 : c0ca2940 [ 818.804750] r7 : 00000004 r6 : c0ca2934 r5 : 00000000 r4 : ffffe000 [ 818.804752] r3 : c0ca2940 r2 : 2aaa8000 r1 : 00000000 r0 : 2aaa8000 [ 818.804754] Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment none [ 818.804756] Control: 10c5387d Table: 6a5f006a DAC: 00000051 [ 818.804758] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.1.0-00002-g192664e-dirty #27 [ 818.804760] Hardware name: Generic R-Car Gen2 (Flattened Device Tree) [ 818.804762] [<c011049c>] (unwind_backtrace) from [<c010bc0c>] (show_stack+0x18/0x1c) [ 818.804764] [<c010bc0c>] (show_stack) from [<c07f7458>] (dump_stack+0x7c/0x9c) [ 818.804766] [<c07f7458>] (dump_stack) from [<c07fd358>] (nmi_cpu_backtrace+0x98/0xb8) [ 818.804768] [<c07fd358>] (nmi_cpu_backtrace) from [<c010e6bc>] (handle_IPI+0x2bc/0x33c) [ 818.804770] [<c010e6bc>] (handle_IPI) from [<c03fa084>] (gic_handle_irq+0x8c/0x98) [ 818.804773] [<c03fa084>] (gic_handle_irq) from [<c0101a8c>] (__irq_svc+0x6c/0x90) [ 818.804774] Exception stack(0xc0d01e68 to 0xc0d01eb0) [ 818.804777] 1e60: 2aaa8000 00000000 2aaa8000 c0ca2940 ffffe000 00000000 [ 818.804779] 1e80: c0ca2934 00000004 c0ca2940 0000000a 00000001 c0d03080 00200102 c0d01eb8 [ 818.804781] 1ea0: c0126aa0 c01021e4 60030113 ffffffff [ 818.804783] [<c0101a8c>] (__irq_svc) from [<c01021e4>] (__do_softirq+0x7c/0x344) [ 818.804785] [<c01021e4>] (__do_softirq) from [<c0126aa0>] (irq_exit+0x6c/0xcc) [ 818.804787] [<c0126aa0>] (irq_exit) from [<c016c344>] (__handle_domain_irq+0x88/0xbc) [ 818.804789] [<c016c344>] (__handle_domain_irq) from [<c03fa058>] (gic_handle_irq+0x60/0x98) [ 818.804791] [<c03fa058>] (gic_handle_irq) from [<c0101a8c>] (__irq_svc+0x6c/0x90) [ 818.804793] Exception stack(0xc0d01f38 to 0xc0d01f80) [ 818.804795] 1f20: 00000000 0008bbb8 [ 818.804798] 1f40: eb74b520 c011aee0 00000001 ffffe000 c0d08c30 c0d8e5be c0d08c74 c0d980c0 [ 818.804800] 1f60: 00000001 c0c5ea40 00000000 c0d01f88 c01084ec c014c85c 60030013 ffffffff [ 818.804802] [<c0101a8c>] (__irq_svc) from [<c014c85c>] (do_idle+0xf0/0x13c) [ 818.804804] [<c014c85c>] (do_idle) from [<c014cb10>] (cpu_startup_entry+0x20/0x24) [ 818.804806] [<c014cb10>] (cpu_startup_entry) from [<c0c00fac>] (start_kernel+0x3e0/0x488) Note: - After I input the second ifconfig eth0 up, I cannot input any command. - If I disconnect an ethernet cable, I can input command again. --- > > To fix the issue, this patch adds a condition and set the RMII mode > > regiseter in sh_eth_dev_exit() for some SoCs. > > > > Note that when I have tried to move the sh_eth_dev_exit() calling > > after phy_stop() on sh_eth_close(), but it gets worse. > > Ah, I was going to suggest changing the call order... what happens then? Sometimes, the following error happened and then I cannot recover the system even if I disconnected an ethernet cable. Do you prefer this way, instead of adding RMII mode setting on sh_eth_dev_exit()? --- # ifconfig eth0 up; [ 64.896021] Micrel KSZ8041RNLI ee700000.ethernet-ffffffff:01: attached PHY driver [Micrel KSZ8041RNLI] (mii_bus:phy_addr=ee700000.ethernet-ffffffff:01, irq=157) # [ 66.868241] sh-eth ee700000.ethernet eth0: Link is Up - 100Mbps/Full - flow control rx/tx [ 66.881777] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready # ifconfig eth0 down [ 71.164900] sh-eth ee700000.ethernet eth0: Link is Down [ 92.169767] rcu: INFO: rcu_sched self-detected stall on CPU [ 92.180654] rcu: 0-....: (2100 ticks this GP) idle=9d6/1/0x40000004 softirq=513/513 fqs=1050 [ 92.194720] rcu: (t=2102 jiffies g=-367 q=46) [ 92.204723] NMI backtrace for cpu 0 [ 92.213705] CPU: 0 PID: 228 Comm: ifconfig Not tainted 5.1.0-00002-g192664e-dirty #28 [ 92.227149] Hardware name: Generic R-Car Gen2 (Flattened Device Tree) [ 92.239258] [<c011049c>] (unwind_backtrace) from [<c010bc0c>] (show_stack+0x18/0x1c) [ 92.252683] [<c010bc0c>] (show_stack) from [<c07f7458>] (dump_stack+0x7c/0x9c) [ 92.265548] [<c07f7458>] (dump_stack) from [<c07fd360>] (nmi_cpu_backtrace+0xa0/0xb8) [ 92.279039] [<c07fd360>] (nmi_cpu_backtrace) from [<c07fd3fc>] (nmi_trigger_cpumask_backtrace+0x84/0x114) [ 92.294384] [<c07fd3fc>] (nmi_trigger_cpumask_backtrace) from [<c01827e4>] (rcu_dump_cpu_stacks+0xb4/0xd0) [ 92.309893] [<c01827e4>] (rcu_dump_cpu_stacks) from [<c0180cc8>] (rcu_sched_clock_irq+0x2cc/0x6f0) [ 92.324744] [<c0180cc8>] (rcu_sched_clock_irq) from [<c0189170>] (update_process_times+0x38/0x64) [ 92.339528] [<c0189170>] (update_process_times) from [<c019a5d0>] (tick_nohz_handler+0xd4/0x128) [ 92.354261] [<c019a5d0>] (tick_nohz_handler) from [<c06573a4>] (arch_timer_handler_virt+0x30/0x38) [ 92.369207] [<c06573a4>] (arch_timer_handler_virt) from [<c0171548>] (handle_percpu_devid_irq+0xf0/0x224) [ 92.384817] [<c0171548>] (handle_percpu_devid_irq) from [<c016c2ac>] (generic_handle_irq+0x20/0x30) [ 92.399910] [<c016c2ac>] (generic_handle_irq) from [<c016c364>] (__handle_domain_irq+0xa8/0xbc) [ 92.414678] [<c016c364>] (__handle_domain_irq) from [<c03fa058>] (gic_handle_irq+0x60/0x98) [ 92.429128] [<c03fa058>] (gic_handle_irq) from [<c0101a8c>] (__irq_svc+0x6c/0x90) [ 92.442748] Exception stack(0xea1f9b50 to 0xea1f9b98) [ 92.453924] 9b40: 2aaa8000 00000000 2aaa8000 c0ca2940 [ 92.468345] 9b60: ffffe000 00000000 c0ca2934 00000002 c0ca2940 0000000a 00000000 c0d03080 [ 92.482764] 9b80: 00404100 ea1f9ba0 c0126aa0 c01021e4 60030113 ffffffff [ 92.495519] [<c0101a8c>] (__irq_svc) from [<c01021e4>] (__do_softirq+0x7c/0x344) [ 92.508973] [<c01021e4>] (__do_softirq) from [<c0126aa0>] (irq_exit+0x6c/0xcc) [ 92.522215] [<c0126aa0>] (irq_exit) from [<c016c344>] (__handle_domain_irq+0x88/0xbc) [ 92.536092] [<c016c344>] (__handle_domain_irq) from [<c03fa058>] (gic_handle_irq+0x60/0x98) [ 92.550513] [<c03fa058>] (gic_handle_irq) from [<c0101a8c>] (__irq_svc+0x6c/0x90) [ 92.564095] Exception stack(0xea1f9c20 to 0xea1f9c68) [ 92.575129] 9c20: f0911320 00000001 00000000 f0911320 00000003 eabb7844 00000001 00000000 [ 92.589271] 9c40: 00000000 00003800 00000000 ea1f9e50 ea1f9d24 ea1f9c70 c0555d80 c055c590 [ 92.603319] 9c60: 60030013 ffffffff [ 92.612529] [<c0101a8c>] (__irq_svc) from [<c055c590>] (sh_mdio_ctrl+0x3c/0x58) [ 92.625531] [<c055c590>] (sh_mdio_ctrl) from [<c0555d80>] (mdiobb_send_num+0x28/0x34) [ 92.638971] [<c0555d80>] (mdiobb_send_num) from [<c0555f10>] (mdiobb_write+0x70/0x94) [ 92.652337] [<c0555f10>] (mdiobb_write) from [<c05555a4>] (__mdiobus_write+0x64/0x10c) [ 92.665688] [<c05555a4>] (__mdiobus_write) from [<c05526e8>] (__phy_modify_changed+0x58/0x6c) [ 92.679587] [<c05526e8>] (__phy_modify_changed) from [<c0552760>] (__phy_modify+0x10/0x18) [ 92.693127] [<c0552760>] (__phy_modify) from [<c05527a4>] (phy_modify+0x3c/0x54) [ 92.705749] [<c05527a4>] (phy_modify) from [<c0553b14>] (phy_suspend+0xb0/0xd0) [ 92.718274] [<c0553b14>] (phy_suspend) from [<c0551118>] (phy_state_machine+0x1a4/0x1a8) [ 92.731589] [<c0551118>] (phy_state_machine) from [<c05511a8>] (phy_stop+0x8c/0xb8) [ 92.744411] [<c05511a8>] (phy_stop) from [<c055e2d4>] (sh_eth_close+0x58/0xa0) [ 92.756668] [<c055e2d4>] (sh_eth_close) from [<c06ce740>] (__dev_close_many+0xc0/0xdc) [ 92.769620] [<c06ce740>] (__dev_close_many) from [<c06d4f9c>] (__dev_change_flags+0xdc/0x1b0) [ 92.783186] [<c06d4f9c>] (__dev_change_flags) from [<c06d5090>] (dev_change_flags+0x20/0x50) [ 92.796660] [<c06d5090>] (dev_change_flags) from [<c074d740>] (devinet_ioctl+0x2d8/0x5dc) [ 92.809892] [<c074d740>] (devinet_ioctl) from [<c074f8f8>] (inet_ioctl+0x260/0x324) [ 92.822560] [<c074f8f8>] (inet_ioctl) from [<c06b2ea4>] (sock_ioctl+0x148/0x424) [ 92.834968] [<c06b2ea4>] (sock_ioctl) from [<c0252e00>] (vfs_ioctl+0x28/0x3c) [ 92.847110] [<c0252e00>] (vfs_ioctl) from [<c0253614>] (do_vfs_ioctl+0x98/0x844) [ 92.859518] [<c0253614>] (do_vfs_ioctl) from [<c0253e00>] (ksys_ioctl+0x40/0x5c) [ 92.871875] [<c0253e00>] (ksys_ioctl) from [<c0101000>] (ret_fast_syscall+0x0/0x54) [ 92.884498] Exception stack(0xea1f9fa8 to 0xea1f9ff0) [ 92.894477] 9fa0: 000ab853 bec4fc90 00000003 00008914 bec4fc90 000ab853 [ 92.907648] 9fc0: 000ab853 bec4fc90 bec4ff52 00000036 bec4fe48 000000b8 00096710 00000000 [ 92.920805] 9fe0: 000ca1e4 bec4fc2c 0001aa8c b6e72f3c --- > > Signed-off-by: Yoshihiro Shimoda <yoshihiro.shimoda.uh@xxxxxxxxxxx> > > --- > > drivers/net/ethernet/renesas/sh_eth.c | 4 ++++ > > 1 file changed, 4 insertions(+) > > > > diff --git a/drivers/net/ethernet/renesas/sh_eth.c b/drivers/net/ethernet/renesas/sh_eth.c > > index e33af37..106ae90 100644 > > --- a/drivers/net/ethernet/renesas/sh_eth.c > > +++ b/drivers/net/ethernet/renesas/sh_eth.c > > @@ -1596,6 +1596,10 @@ static void sh_eth_dev_exit(struct net_device *ndev) > > > > /* Set MAC address again */ > > update_mac_address(ndev); > > + > > + /* Set the mode again if required */ > > Should be "RMII mode", n ot just "Mode". We prolly need more detailed explanation... I got it. Best regards, Yoshihiro Shimoda > > + if (mdp->cd->rmiimode) > > + sh_eth_write(ndev, 0x1, RMIIMODE); > > } > > > > static void sh_eth_rx_csum(struct sk_buff *skb) > > MBR, Sergei
