On 4/9/24 07:28, Jiri Pirko wrote:
Tue, Apr 09, 2024 at 03:05:47PM CEST, f.fainelli@xxxxxxxxx wrote:
On 4/9/2024 3:56 AM, Jiri Pirko wrote:
Mon, Apr 08, 2024 at 11:36:42PM CEST, f.fainelli@xxxxxxxxx wrote:
On 4/8/24 09:51, Jiri Pirko wrote:
Mon, Apr 08, 2024 at 05:46:35PM CEST, alexander.duyck@xxxxxxxxx wrote:
On Mon, Apr 8, 2024 at 4:51 AM Jiri Pirko <jiri@xxxxxxxxxxx> wrote:
Fri, Apr 05, 2024 at 08:38:25PM CEST, alexander.duyck@xxxxxxxxx wrote:
On Fri, Apr 5, 2024 at 8:17 AM Jason Gunthorpe <jgg@xxxxxxxxxx> wrote:
On Fri, Apr 05, 2024 at 07:24:32AM -0700, Alexander Duyck wrote:
Alex already indicated new features are coming, changes to the core
code will be proposed. How should those be evaluated? Hypothetically
should fbnic be allowed to be the first implementation of something
invasive like Mina's DMABUF work? Google published an open userspace
for NCCL that people can (in theory at least) actually run. Meta would
not be able to do that. I would say that clearly crosses the line and
should not be accepted.
Why not? Just because we are not commercially selling it doesn't mean
we couldn't look at other solutions such as QEMU. If we were to
provide a github repo with an emulation of the NIC would that be
enough to satisfy the "commercial" requirement?
My test is not "commercial", it is enabling open source ecosystem vs
benefiting only proprietary software.
Sorry, that was where this started where Jiri was stating that we had
to be selling this.
For the record, I never wrote that. Not sure why you repeat this over
this thread.
Because you seem to be implying that the Meta NIC driver shouldn't be
included simply since it isn't going to be available outside of Meta.
The fact is Meta employs a number of kernel developers and as a result
of that there will be a number of kernel developers that will have
access to this NIC and likely do development on systems containing it.
In addition simply due to the size of the datacenters that we will be
populating there is actually a strong likelihood that there will be
more instances of this NIC running on Linux than there are of some
other vendor devices that have been allowed to have drivers in the
kernel.
So? The gain for community is still 0. No matter how many instances is
private hw you privately have. Just have a private driver.
I am amazed and not in a good way at how far this has gone, truly.
This really is akin to saying that any non-zero driver count to maintain is a
burden on the community. Which is true, by definition, but if the goal was to
build something for no users, then clearly this is the wrong place to be in,
or too late. The systems with no users are the best to maintain, that is for
sure.
If the practical concern is wen you make tree wide API change that fbnic
happens to use, and you have yet another driver (fbnic) to convert, so what?
Work with Alex ahead of time, get his driver to be modified, post the patch
series. Even if Alex happens to move on and stop being responsible and there
is no maintainer, so what? Give the driver a depreciation window for someone
to step in, rip it, end of story. Nothing new, so what has specifically
changed as of April 4th 2024 to oppose such strong rejection?
How you describe the flow of internal API change is totally distant from
reality. Really, like no part is correct:
1) API change is responsibility of the person doing it. Imagine working
with 40 driver maintainers for every API change. I did my share of
API changes in the past, maintainer were only involved to be cced.
As a submitter you propose changes and silence is acknowledgement. If one of
your API changes broke someone's driver and they did not notify you of the
breakage during the review cycle, it falls on their shoulder to fix it for
themselves and they should not be holding back your work, that would not be
Does it? I don't think so. If you break something, better try to fix it
before somebody else has to.
fair. If you know about the breakage, and there is still no fix, that is an
indication the driver is not actively used and maintained.
So? That is not my point. If I break something in fbnic, why does anyone
care? Nobody is ever to hit that bug, only Meta DC.
They care, and they will jump in to fix it. There is no expectation that
as a community member you should be able to make 100% correct patches,
this is absolutely not humanly possible, even less so with scarce access
to the hardware. All you can hope for is that your changes work, and
that someone catches it, sooner rather than later.
This also does not mean you have to do the entire API changes to a driver you
do not know about on your own. Nothing ever prevents you from posting the
patches as RFC and say: "here is how I would go about changing your driver,
please review and help me make corrections". If the driver maintainers do not
respond there is no reason their lack of involvement should refrain your
work, and so your proposed changes will be merged eventually.
Realistically, did you see that ever happen. I can't recall.
This happens all of the time, if you make a netdev tree wide change, how
many maintainer's Acked-by do we collect before merging those changes:
none typically because some netdev maintainers are just quicker than
reviewers could be. In other subsystems we might actually wait for
people to give a change to give their A-b or R-b tags, not always though.
Is not this the whole point of being a community and be able to delegate and
mitigate the risk of large scale changes?
2) To deprecate driver because the maintainer is not responsible. Can
you please show me one example when that happened in the past?
I cannot show you an example because we never had to go that far and I did
not say that this is an established practice, but that we *could* do that if
we ever reached that point.
You are talking about a flow that does not exist. I don't understand how
is that related to this discussion then.
I was trying to appease your concerns about additional maintenance
burden. If the burden becomes real, we ditch it. We can dismiss this
point as being not relevant if you want.
Like it was said, there are tons of drivers in the Linux kernel that have a
single user, this one might have a few more than a single one, that should be
good enough.
This will have exactly 0. That is my point. Why to merge something
nobody will ever use?
Even if Alex and his firmware colleague end up being the only two people
using this driver if the decision is to make it upstream because this is the
desired distribution and development model of the driver we should respect
that.
And just to be clear, we should not be respecting that because Meta, or Alex
or anyone decided that they were doing the world a favor by working in the
open rather than being closed door, but simply because we cannot *presume*
I don't see any favor for the community. What's the favor exactly?
There is no exchange of favors or "this" for "that", this is not how a
community works. You bring your code to the table, solicit review
feedback, then go on to maintain it within your bounds, and, time
permitting, beyond your driver. What we gain as a community is
additional visibility, more API users (eventually real world users,
too), and therefore a somewhat more objective way of coming up with new
APIs and features, and just a broader understanding of what is out
there. This is better than speculation since that creates a less skewed
mental model.
Let us say that someone at Meta wanted to get this core netdev feature
that could be super cool for others included in the upstream kernel, we
would shut it down on the basis that no user exists and we would be
right about doing it that. Turns out there is a user, but the driver
lives out of tree, but now we also reject that driver? Who benefits from
doing that: nobody.
You need a membership card to join the club that you can only enter if
you have a membership card already? No thank you.
The only favor I see is the in the opposite direction, community giving
Meta free cycles saving their backporting costs. Why?
Technically it would be both forward porting cycles, since they would no
longer need to rebase the driver against their most recent kernel used,
and backporting cycles for the first kernel including fbnic onwards.
That comes almost for free these days anyways thanks to static analysis
tools. The overwhelming cost of the maintenance remains on Meta
engineers, being the only ones with access to the hardware. If they end
up with customers in the future, they can offload some of that to their
customers, too.
Let's just look at a few high profile drivers by lines changed:
Summary for: drivers/net/ethernet/mellanox/mlxsw
Total: 133422 (+), 44952 (-)
Own: 131180 (+), 42725 (-)
Community: 2242 (+) (1.680 %), 2227 (-) (4.954 %)
Summary for: drivers/net/ethernet/mellanox/mlx5
Total: 265368 (+), 107690 (-)
Own: 259213 (+), 100328 (-)
Community: 6155 (+) (2.319 %), 7362 (-) (6.836 %)%
Summary for: drivers/net/ethernet/broadcom/bnxt
Total: 70355 (+), 25402 (-)
Own: 68458 (+), 23358 (-)
Community: 1897 (+) (2.696 %), 2044 (-) (8.047 %)
Summary for: drivers/net/ethernet/intel/e1000e/
Total: 39760 (+), 9924 (-)
Own: 38514 (+), 8905 (-)
Community: 1246 (+) (3.134 %), 1019 (-) (10.268 %)
I admit this is simplistic because both mlxsw and mlx5 drivers helped
greatly improve the networking stack in parts that I benefited directly
from within DSA for instance.
The point is, you paid the maintenance price though, the community did not.
about their intentions and the future.
Heh, the intention is pretty clear from this discussion, isn't it? If
they ever by any chance decide to go public with their device, driver
for that could be submitted at a time. But this is totally hypothetical.
I think your opposition is unreasonable and is unfair. Using your
argument to the extreme, I may go as far as saying that it encourages
working out of tree, rather than in tree. This is the exact opposite of
what made Linux successful as an OS.
Can I buy a Spectrum switch off Amazon?
--
Florian
