On 6/27/22 07:34, Zbigniew Jędrzejewski-Szmek wrote:
> On Mon, Jun 27, 2022 at 09:40:53AM +0100, Daniel P. Berrangé wrote:
>> On Mon, Jun 27, 2022 at 10:10:31AM +0200, Zbigniew Jędrzejewski-Szmek wrote:
>>> On Sun, Jun 26, 2022 at 12:36:14AM +0530, Vipul Siddharth wrote:
>>>> This document represents a proposed Change. As part of the Changes
>>>> process, proposals are publicly announced in order to receive
>>>> community feedback. This proposal will only be implemented if approved
>>>> by the Fedora Engineering Steering Committee.
>>>>
>>>>
>>>> == Summary ==
>>>>
>>>> The `systemd-udev` package installs
>>>> `"/usr/lib/systemd/network/99-default.link"`,
>>>> which sets `Link.MACAddressPolicy=persistent`. This proposal is to
>>>> change it to set `Link.MACAddressPolicy=none` to stop changing the MAC address.
>>>> This is particularly important for bridge and bond devices.
>>>>
>>>> This change can either only apply to bridge/bond devices, or to
>>>> various software devices. That is to be discussed.
>>>
>>> Hi!
>>>
>>> I already participated in the upstream discussion, so what I write
>>> here will be a restatement to some extent, but with a look from the
>>> side of Fedora specifically.
>>>
>>> The proposal has two variants: 1. just changing the policy to MACAddressPolicy=none
>>> or 2. limiting the change to bridge and bond devices.
>>>
>>> Re variant 1: MACAddressPolicy=persistent applies to all devices that don't have
>>> a hardware address. The proposal as written (blanket MACAddressPolicy=none)
>>> would change behaviour for all kinds of devices, incl. e.g. software
>>> devices like veths, and cheap hardware devices without a fixed MAC.
>>> The proposal doesn't provide any justification for this (except for
>>> simplicity of implementation) and this variant seems pretty bad and
>>> I'm strongly opposed.
>>>
>>> Re variant 2: the proposal limited to brige/bond devices seems much more
>>> reasonable. In particular, the case described below of a server (virtualized
>>> or not) in a big datacenter is the one case where the benefits of
>>> MACAddressPolicy=none are clearly visible. I still don't think it's worth
>>> changing the default, but here the cost:benefit ratio is much closer.
>>>
>>>> == Benefit to Fedora ==
>>>>
>>>> Pros:
>>>>
>>>> - Consistent behavior with RHEL8 and RHEL9.
>>>>
>>>> - Avoid race of udev and the tool that creates the interface.
>>>
>>> The race will happen if the creation is done in a specific way. But at
>>> the same time, even the Change proposal describes how to avoid the
>>> race ('ip link add address aa:aa:aa:aa:aa:aa type bridge'). So the situation
>>> can be summarized as "we have a bunch of 'big' tools that create
>>> devices like NetworkManager or systemd-networkd, which already know or
>>> can be easily fixed to avoid the race, and a manual tool which can be invoked
>>> in a way that avoids the race". Instead of changing the default in udev we
>>> could educate people how to invoke it better.
>>
>> This comes across as blaming every networking tool out there for
>> having their previously correct & working behaviour be broken by
>> a systemd change imposing new requirements on them.
>
> I was a bit sloppy in my phrasing. NetworkManager and systemd-networkd
> already do the right thing. 'ip link' is the odd one out. Various other
> tools (e.g. netplan or Debian's network scripts) don't implement this internally,
> but call e.g. NetworkManager or 'ip'. So if we changed 'ip', this would go
> a long way to solving the problem (I'm may be wrong on some details here,
> please correct me if necessary.)
>
> I'm not "blaming" the tools, I completely understand that they were
> written a long time ago. But in fact the issue is fairly generic: any
> software which interacts with devices that udev also touches MUST wait
> for udev to be done with the device. It's not just the MAC address
> policy, but also other rules that users may configure locally, sysctl
> configuration, etc. Without synchronization, one runs into races and
> errors from the tools when they try to configure things in parallel.
>
>> Are there any notable tools which actually follow the usage pattern
>> described, or are we in effect having to fix *everything*, including
>> an uncountable amount of documentation, blogs, examples, most of
>> which will prove impossible to fix in practice.
>>
>>>> - Bridge and bond interfaces can get the MAC addresses from their first port.
>>>>
>>>> In the case of `MACAddressPolicy=none` for a bond (or bridge) the bond will
>>>> get a MAC related to one of its physical NIC devices. In the case of
>>>> provisioning
>>>> new systems (especially in a large datacenter) information about the server
>>>> can be used to configure the network environment (DHCP, Firewall, etc) before
>>>> the server is ever even powered on. This does mean that you may have to update
>>>> your network environment configuration if you replace a NIC (con), but that you
>>>> won't have to update your network environment configuration if you re-install
>>>> your server (pro), which is what you'd have to do now with
>>>> `MACAddressPolicy=persistent`.
>>>
>>> Yep. This is *the* case.
>>
>> Having the bridge/bond get the MAC addr of tht first physical NIC is pretty
>> compelling from an automation POV, especially when the virtualziation host is
>> doing traffic filtering for the VM. When a mgmt app assigns a MAC & IP pair to
>> a guest, it is not uncommon to apply firewall rules providing anti-spoofing
>> protection for MAC/IP, such that the VM cannot send traffic with other MAC/IP
>> addresses. Libvirt provides this functionality with its 'clean-traffic'
>> network filter, and other virt/cloud mgmt apps do similar.
>>
>>>> Cons:
>>>>
>>>> - Deviate from upstream systemd.
>>
>> This is disappointing, as a great benefit of systemd is the level of
>> consistency it brought to distro behaviour, and I think this change
>> would be useful to all distros (in the context of bridge/bond devices)
>
> That is a good point. If this is done, I too would prefer to change it
> upstream rather than just downstream in Fedora.
Big +1 from me on getting upstream to change so that we are in alignment. I don't
know if it was made clear in the upstream discussion, but maybe upstream would be
more accepting for just changing the policy for bond/bridge/team devices.
Similarly, we might be able to get the next RHEL to adopt the approach of
MACAddressPolicy=persistent for non bond/bridge/team and MACAddressPolicy=none
for bond/bridge/team. IOW we may be able to get upstream systemd, Fedora, and
downstream (RHEL/CentOS,Alma,Rocky) to all be in alignment.
Dusty
_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
