On Thu, Sep 02, 2021 at 11:24:39PM +0100, Russell King (Oracle) wrote: > On Fri, Sep 03, 2021 at 12:39:49AM +0300, Vladimir Oltean wrote: > > On Thu, Sep 02, 2021 at 10:33:03PM +0100, Russell King (Oracle) wrote: > > > That's probably an unreliable indicator. DPAA2 has weirdness in the > > > way it can dynamically create and destroy network interfaces, which > > > does lead to problems with the rtnl lock. I've been carrying a patch > > > from NXP for this for almost two years now, which NXP still haven't > > > submitted: > > > > > > http://git.armlinux.org.uk/cgit/linux-arm.git/commit/?h=cex7&id=a600f2ee50223e9bcdcf86b65b4c427c0fd425a4 > > > > > > ... and I've no idea why that patch never made mainline. I need it > > > to avoid the stated deadlock on SolidRun Honeycomb platforms when > > > creating additional network interfaces for the SFP cages in userspace. > > > > Ah, nice, I've copied that broken logic for the dpaa2-switch too: > > https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git/commit/?id=d52ef12f7d6c016f3b249db95af33f725e3dd065 > > > > So why don't you send the patch? I can send it too if you want to, one > > for the switch and one for the DPNI driver. > > Sorry, I mis-stated. NXP did submit that exact patch, but it's actually > incorrect for the reason I stated when it was sent: > > https://patchwork.ozlabs.org/project/netdev/patch/1574363727-5437-2-git-send-email-ioana.ciornei@xxxxxxx/ So why are you carrying it then? > I did miss the rtnl_lock() around phylink_disconnect_phy() in the > description of the race, which goes someway towards hiding it, but > there is still a race between phylink_destroy() and another thread > calling dpaa2_eth_get_link_ksettings(), and priv->mac being freed: > > static int > dpaa2_eth_get_link_ksettings(struct net_device *net_dev, > struct ethtool_link_ksettings *link_settings) > { > struct dpaa2_eth_priv *priv = netdev_priv(net_dev); > > if (dpaa2_eth_is_type_phy(priv)) > return phylink_ethtool_ksettings_get(priv->mac->phylink, > link_settings); > > which dereferences priv->mac and priv->mac->phylink, vs: > > static irqreturn_t dpni_irq0_handler_thread(int irq_num, void *arg) > { > ... > if (status & DPNI_IRQ_EVENT_ENDPOINT_CHANGED) { > dpaa2_eth_set_mac_addr(netdev_priv(net_dev)); > dpaa2_eth_update_tx_fqids(priv); > > if (dpaa2_eth_has_mac(priv)) > dpaa2_eth_disconnect_mac(priv); > else > dpaa2_eth_connect_mac(priv); > } > > static void dpaa2_eth_disconnect_mac(struct dpaa2_eth_priv *priv) > { > if (dpaa2_eth_is_type_phy(priv)) > dpaa2_mac_disconnect(priv->mac); > > if (!dpaa2_eth_has_mac(priv)) > return; > > dpaa2_mac_close(priv->mac); > kfree(priv->mac); <== potential use after free bug by > priv->mac = NULL; <== dpaa2_eth_get_link_ksettings() > } Okay, so this needs to stay under the rtnetlink mutex, to serialize with dpaa2_eth_get_link_ksettings which is already under the rtnetlink mutex. So the way in which rtnl_lock is taken right now is actually fine in a way. > > void dpaa2_mac_disconnect(struct dpaa2_mac *mac) > { > if (!mac->phylink) > return; > > phylink_disconnect_phy(mac->phylink); > phylink_destroy(mac->phylink); <== another use-after-free bug via > dpaa2_eth_get_link_ksettings() > dpaa2_pcs_destroy(mac); > } > > Note that phylink_destroy() is documented as: > > * Note: the rtnl lock must not be held when calling this function. > > because it calls sfp_bus_del_upstream(), which will take the rtnl lock > itself. An alternative solution would be to remove the rtnl locking > from sfp_bus_del_upstream(), but then force _everyone_ to take the > rtnl lock before calling phylink_destroy() - meaning a larger block of > code ends up executing under the lock than is really necessary. So phylink_destroy has exactly 20 call sites, it is not that bad? And as for "larger block than necessary" - doesn't the dpaa2 prolonged usage count as necessary? > However, as I stated in my review of the patch "As I've already stated, > the phylink is not designed to be created and destroyed on a published > network device." That still remains true today, and it seems that the > issue has never been fixed in DPAA2 despite having been pointed out. So what would you do, exactly, to "fix" the issue that a DPNI can connect and disconnect at runtime from a DPMAC? Also, "X is not designed to Y" doesn't really say much, given a bit of will power. Linux was not designed to run on non-i386 either. Any other issues besides needing to take rtnl_mutex top-level when calling phylink_destroy? Since phylink_disconnect_phy needs it anyway, and phylink_destroy ends up calling sfp_bus_del_upstream which takes the same mutex again, and drivers that connect/disconnect at probe/remove time end up calling both in a row, I don't think there is much of an issue to speak of, or that the rework would be that difficult.
