On 1/17/23 1:22 PM, Toke Høiland-Jørgensen wrote:
Daniel Borkmann <daniel@xxxxxxxxxxxxx> writes:
On 1/17/23 12:30 PM, Toke Høiland-Jørgensen wrote:
Daniel Borkmann <daniel@xxxxxxxxxxxxx> writes:
On 1/16/23 11:57 PM, Toke Høiland-Jørgensen wrote:
Following up on the discussion at the BPF office hours, this patch adds a
description of the (new) concept of "stable kfuncs", which are kfuncs that
offer a "more stable" interface than what we have now, but is still not
part of UAPI.
This is mostly meant as a straw man proposal to focus discussions around
stability guarantees. From the discussion, it seemed clear that there were
at least some people (myself included) who felt that there needs to be some
way to export functionality that we consider "stable" (in the sense of
"applications can rely on its continuing existence").
One option is to keep BPF helpers as the stable interface and implement
some technical solution for moving functionality from kfuncs to helpers
once it has stood the test of time and we're comfortable committing to it
as a stable API. Another is to freeze the helper definitions, and instead
use kfuncs for this purpose as well, by marking a subset of them as
"stable" in some way. Or we can do both and have multiple levels of "stable",
I suppose.
This patch is an attempt to describe what the "stable kfuncs" idea might look
like, as well as to formulate some criteria for what we mean by "stable", and
describe an explicit deprecation procedure. Feel free to critique any part
of this (including rejecting the notion entirely).
Some people mentioned (in the office hours) that should we decide to go in
this direction, there's some work that needs to be done in libbpf (and
probably the kernel too?) to bring the kfunc developer experience up to par
with helpers. Things like exporting kfunc definitions to vmlinux.h (to make
them discoverable), and having CO-RE support for using them, etc. I kinda
consider that orthogonal to what's described here, but I added a
placeholder reference indicating that this (TBD) functionality exists.
Thanks for the writeup.. I did some edits to your sections to make some parts
more clear and to leave out other parts (e.g. libbpf-related bits which are not
relevant in here and it's one of many libs). I also edited some parts to leave
us more flexibility. Here would be my take mixed in:
Edits LGTM, with just one nit, below:
3. API (in)stability of kfuncs
==============================
By default, kfuncs exported to BPF programs are considered a kernel-internal
interface that can change between kernel versions. In the extreme case that
could also include removal of a kfunc. This means that BPF programs using
kfuncs might need to adapt to changes between kernel versions. In other words,
kfuncs are _not_ part of the kernel UAPI! Rather, these kfuncs can be thought
of as being similar to internal kernel API functions exported using the
``EXPORT_SYMBOL_GPL`` macro. All new BPF kernel helper-like functionality must
initially start out as kfuncs.
3.1 Promotion to "stable"
-------------------------
While kfuncs are by default considered unstable as described above, some kfuncs
may warrant a stronger stability guarantee and could be marked as *stable*. The
decision to move a kfunc to *stable* is taken on a case-by-case basis and has
a high barrier, taking into account its usefulness under longer-term production
deployment without any unforeseen API issues or limitations. In general, it is
Forgot, we should probably also add after "[...] or limitations.":
Such promotion request along with aforementioned argumentation on why a kfunc
is ready to be stabilized must be driven from developer-side.
What does "driven from developer-side" mean, exactly? And what kind of
developers (BPF app developers, or kernel devs)?
Mainly to denote that this needs to be an explicit request from the community
rather than something that would happen automagically after some time (e.g.
where maintainers would just put the KF_STABLE stamp to it). 'kfunc xyz has
been used in our fleet in production in the context of project abc for two
years now and its API is sufficient to cover all foreseeable needs. The
kfunc didn't need to get extended since it was added [...]', for example.
The developer-hat can be both as long as there is a concrete relation to
usage of the kfunc that can be provided to then make the case.
not expected that every kfunc will turn into a stable one - think of it as an
exception rather than the norm. kfuncs which have been promoted to stable are
then marked using the ``KF_STABLE`` tag. The possibility from a stable kfunc to
a BPF helper addition is up to the maintainers to decide.
1. Stable kfuncs will not change their function signature or functionality in
a way that may cause incompatibilities for BPF programs calling the function.
2. The BPF community will make every reasonable effort to keep stable kfuncs
around as long as they continue to be useful to real-world BPF applications.
3. Should a stable kfunc turn out to be no longer useful, a deprecation procedure
might be implemented for them as outlined below.
"deprecation procedure might be implemented" could be interpreted as "we
may implement a deprecation procedure, or we may just remove it without
one". Which is presumably not what you meant? So maybe:
3. Should a stable kfunc turn out to be no longer useful, the BPF
community may decide to eventually remove it. In this case, before
being removed that kfunc will go through a deprecation procedure as
outlined below.
Yes, that sounds good to me.
Awesome!
-Toke
