Re: Should `MACAddressPolicy=persistent` for bridges/bonds/all-software-devices be reconsidered?

[Dan Streetman <ddstreet@xxxxxxxx>]

On Thu, May 12, 2022 at 2:27 PM Dan Streetman <ddstreet@xxxxxxxx> wrote:
>
> On Thu, May 12, 2022 at 1:50 PM Dusty Mabe <dusty@xxxxxxxxxxxxx> wrote:
> >
> >
> >
> > On 5/12/22 13:36, Dan Streetman wrote:
> > > On Thu, May 12, 2022 at 11:11 AM Thomas Haller <thaller@xxxxxxxxxx> wrote:
> > >>
> > >> Hi Zbyszek,
> > >>
> > >>
> > >>
> > >> I must say, I personally don't care too much. NetworkManager is fine
> > >> either way.
> > >>
> > >> There is however the problem about RHEL8/9, which patches this
> > >> downstream. I don't like that deviation, but I'd also be uncomfortable
> > >> to push that change for RHEL(10) users.
> > >>
> > >> But let me play devil's advocate here...
> > >>
> > >>
> > >> On Mon, 2022-05-09 at 19:27 +0200, Zbigniew Jędrzejewski-Szmek wrote:
> > >>>
> > >>> FWIW, I still think it's a better _default_.
> > >>
> > >> I don't agree that it's clearly better. Your arguments don't seem
> > >> strong, arguably, neither are mine. Except, that it's not clear that
> > >> this solves an actual problem, while it clearly causes problems for
> > >> some people. Just look at the referened issues from !3374.
> > >>
> > >>
> > >> Either
> > >>
> > >>   - a user doesn't care about the MAC address,
> > >
> > > note that it's possible for a user not to care about the *specific*
> > > mac address, only that they want the mac to remain consistent.
> > >
> > > for example, on my system i have br0 bridge with multiple interfaces
> > > attached, and my local DHCP server is configured to provide a static
> > > addr to br0's mac. If I replace an interface card (e.g. change from 1g
> > > to 10g, or just replace a failing nic) then the br0 mac *does not*
> > > change, if using systemd's default. If I had br0 inheriting its mac
> > > from one of the attached interfaces, it would change, and i'd have to
> > > update my dhcp server config.
> > >
> > > I think the argument really comes down to, should the default be to
> > > have (without user configuration) a mac that is predictable or a mac
> > > that is consistent. Your argument is for predictability, which makes
> > > the field engineer's work deploying systems easier, but if you lose
> > > consistency then the life of the maintenance engineer gets harder.
> >
> > Interested discussion. Let's take it a bit further.
> >
> > On your system how did your DHCP server get configured to provide an
> > addr to br0's MAC? You had to install the OS, and create br0 first
> > before you even knew the MAC to tell the network admin what the MAC
> > was. Now you're golden, the MAC will never change for the lifetime
> > of that OS install even if someone replaces a NIC, but it wasn't a great
> > first experience really.
>
> No, but the alternative is for me to manually find the mac address
> labels on my nics and use those to pre-configure my DHCP server. I
> understand for large deployments those are typically provided in a
> spreadsheet, but that isn't the case for me, and my approach (i.e.
> simply check the mac once the system is installed) was *much* easier
> and more reliable.
>
> And this doesn't even get into the nuances of figuring out *which* nic
> mac the bridge/bond will get, which won't be obvious to everyone. Note
> that the answer here *is not* that the bridge gets the mac of the
> 'first' interface added to it; the bridge gets the mac of the *lowest*
> mac that is currently attached to it. And yes, this does mean that if
> you use this behavior (of a bridge getting its mac from the attached
> interfaces), the bridge's mac *can change while it is up and running
> with an address*. Try it out yourself ;-)

And to clarify - bond interfaces do 'keep' the mac of the 'first'
interface added - technically, the bond sets all interfaces added
later to have the same mac. So this is even less 'predictable' than a
bridge, where you can predict which of the 2 or more macs are
'lowest', a bond mac would just be whatever the 'first' added
interface is.

>
>
> >
> > On the other hand, what if you re-provision that server often (new machine-id)
> > you get a new MAC and you get to dance with your network admin again. OR
> > what if you have disk failure? You most likely backed up your critical data,
> > but did you backup your machine-id that hashes your new MAC? Probably not
> > and even if you did would you want to duplicate that machine-id to the new
> > install you would do?
> >
> > Barring other reasons, if we simplify it down to just the consistency argument,
> > one approach seems better for if you replace NIC cards often and one of them
> > seems better if you re-install your OS often.
>
> Yes, 'consistency' has to be based on something, so 'consistency'
> based on hardware is one approach; consistency based on software is
> another. As you said some users may have software churn and so would
> want to base their 'consistency' on their (presumably unchanging)
> hardware. However, software churn is pretty rare at the bare metal
> level, isn't it? Not unheard of, for sure, but I'd hardly say most
> datacenters redeploy their bare metal servers regularly (though I
> admit I have no real knowledge of how often that happens). If we're
> talking about virtual software churn, which I think is much more
> common (e.g. redeploying an os into a VM), then - assuming the actual
> VM definition doesn't change (because if it did, then there is no hw
> consistency to base the mac consistency on), the machine-id will
> re-use the VM's uuid (provided via DMI as previously mentioned) and so
> the systemd-generated mac won't change.
>
>
> >
> > Dusty
> >
> >