On Mon, Jun 27, 2022 at 10:10:31AM +0200, Zbigniew Jędrzejewski-Szmek wrote:
> On Sun, Jun 26, 2022 at 12:36:14AM +0530, Vipul Siddharth wrote:
> > This document represents a proposed Change. As part of the Changes
> > process, proposals are publicly announced in order to receive
> > community feedback. This proposal will only be implemented if approved
> > by the Fedora Engineering Steering Committee.
> >
> >
> > == Summary ==
> >
> > The `systemd-udev` package installs
> > `"/usr/lib/systemd/network/99-default.link"`,
> > which sets `Link.MACAddressPolicy=persistent`. This proposal is to
> > change it to set `Link.MACAddressPolicy=none` to stop changing the MAC address.
> > This is particularly important for bridge and bond devices.
> >
> > This change can either only apply to bridge/bond devices, or to
> > various software devices. That is to be discussed.
>
> Hi!
>
> I already participated in the upstream discussion, so what I write
> here will be a restatement to some extent, but with a look from the
> side of Fedora specifically.
>
> The proposal has two variants: 1. just changing the policy to MACAddressPolicy=none
> or 2. limiting the change to bridge and bond devices.
>
> Re variant 1: MACAddressPolicy=persistent applies to all devices that don't have
> a hardware address. The proposal as written (blanket MACAddressPolicy=none)
> would change behaviour for all kinds of devices, incl. e.g. software
> devices like veths, and cheap hardware devices without a fixed MAC.
> The proposal doesn't provide any justification for this (except for
> simplicity of implementation) and this variant seems pretty bad and
> I'm strongly opposed.
>
> Re variant 2: the proposal limited to brige/bond devices seems much more
> reasonable. In particular, the case described below of a server (virtualized
> or not) in a big datacenter is the one case where the benefits of
> MACAddressPolicy=none are clearly visible. I still don't think it's worth
> changing the default, but here the cost:benefit ratio is much closer.
>
> > == Benefit to Fedora ==
> >
> > Pros:
> >
> > - Consistent behavior with RHEL8 and RHEL9.
> >
> > - Avoid race of udev and the tool that creates the interface.
>
> The race will happen if the creation is done in a specific way. But at
> the same time, even the Change proposal describes how to avoid the
> race ('ip link add address aa:aa:aa:aa:aa:aa type bridge'). So the situation
> can be summarized as "we have a bunch of 'big' tools that create
> devices like NetworkManager or systemd-networkd, which already know or
> can be easily fixed to avoid the race, and a manual tool which can be invoked
> in a way that avoids the race". Instead of changing the default in udev we
> could educate people how to invoke it better.
This comes across as blaming every networking tool out there for
having their previously correct & working behaviour be broken by
a systemd change imposing new requirements on them.
Are there any notable tools which actually follow the usage pattern
described, or are we in effect having to fix *everything*, including
an uncountable amount of documentation, blogs, examples, most of
which will prove impossible to fix in practice.
> > - Bridge and bond interfaces can get the MAC addresses from their first port.
> >
> > In the case of `MACAddressPolicy=none` for a bond (or bridge) the bond will
> > get a MAC related to one of its physical NIC devices. In the case of
> > provisioning
> > new systems (especially in a large datacenter) information about the server
> > can be used to configure the network environment (DHCP, Firewall, etc) before
> > the server is ever even powered on. This does mean that you may have to update
> > your network environment configuration if you replace a NIC (con), but that you
> > won't have to update your network environment configuration if you re-install
> > your server (pro), which is what you'd have to do now with
> > `MACAddressPolicy=persistent`.
>
> Yep. This is *the* case.
Having the bridge/bond get the MAC addr of tht first physical NIC is pretty
compelling from an automation POV, especially when the virtualziation host is
doing traffic filtering for the VM. When a mgmt app assigns a MAC & IP pair to
a guest, it is not uncommon to apply firewall rules providing anti-spoofing
protection for MAC/IP, such that the VM cannot send traffic with other MAC/IP
addresses. Libvirt provides this functionality with its 'clean-traffic'
network filter, and other virt/cloud mgmt apps do similar.
> > Cons:
> >
> > - Deviate from upstream systemd.
This is disappointing, as a great benefit of systemd is the level of
consistency it brought to distro behaviour, and I think this change
would be useful to all distros (in the context of bridge/bond devices)
> It is also important to mention that Fedora will "deviate" from itself
> (it's former self). We would be changing a default in place since ~2013 [1].
>
> [1] https://github.com/systemd/systemd/commit/16b9b87aee
Yes, that is quite unfortunate.
> > It is desirable that RHEL and Fedora behaves similar. A possible outcome
> > could be the current behavior stays and RHEL 10 would change behavior. On the
> > other hand, different distributions (or even Fedora spins) have different
> > uses and needs. Deviating might be fine. In the same vein, there is also
> > a desire to stay close to upstream systemd behavior. But the uses of systemd
> > project go beyond Fedora/RHEL, so deviating here may also be fine.
>
> So:
> - Variant 1 is not good, variant 2 makes more sense.
> - The motivating case for v.2 is the "big datacenter" case and race
> when 'ip' is invoked.
IMHO The more compelling reason is compliance with MAC address filtering
applied to VMs.
> - We could improve the tools: 'ip link' could be taught to wait until
> udev has stopped processing the device, users can be taught to use
> better invocations.
Teaching users is a impractical solution IMHO. The 'ip' tool already
has poor usability/learnability IME, such that getting users to learn
differences is pretty challenging, plus there's way too much docs out
there in the wild that will never be updatable.
> - For "small" users (individual machine admins who just install some
> server and configure it after turning it on and/or swap network
> cards between machines), having stable MAC addresses that doesn't
> depend on the order of device discovery or removal of a single
> network card seems better.
> - For "big" users (the datacenter case), changing the policy make sense,
> but at the same time, those folks can just insert a policy override,
> they're most likely using some ansible/puppet/cheffy thingy.
> - RHEL is more of the "big" case, Fedora is more the "small" case.
> Sometimes RHEL and Fedora have different defaults, sometimes RHEL
> takes years to follow what Fedora does.
>
> So overall, I think this proposal would make most sense when limited
> to specific types of hardware (bridge and bond?), and also to specific
> spins: it's a better fit for CoreOS than Workstation…
This doesn't feel like a spin-specific problem to me, given it appears
useful to both host & guest scenarios alike, and nested VM means
host+guest are often the same thing.
> But I haven't seen discussion of other approaches: would making 'ip'
> better be an option? 'udev' already locks some devices when processing
> using a simple BSD lock. Should we do the same for network devices
> and just teach 'ip' to do the same? (This would be much simpler and
> maybe more appealing that teaching it to wait for udev to report
> that it's finished).
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
