On 5/28/20 12:15 PM, Alexei Starovoitov wrote:
On Thu, May 28, 2020 at 11:07:09AM -0600, Shuah Khan wrote:
On 5/28/20 10:14 AM, Alexei Starovoitov wrote:
On Thu, May 28, 2020 at 12:56:31PM +0200, Greg KH wrote:
On Thu, May 28, 2020 at 10:05:57AM +0200, Jiri Benc wrote:
On Wed, 27 May 2020 15:23:13 -0700, Alexei Starovoitov wrote:
I prefer to keep selftests/bpf install broken.
This forced marriage between kselftests and selftests/bpf
never worked well. I think it's a time to free them up from each other.
Alexei, it would be great if you could cooperate with other people
instead of pushing your own way. The selftests infrastructure was put
to the kernel to have one place for testing. Inventing yet another way
to add tests does not help anyone. You don't own the kernel. We're
community, we should cooperate.
I agree, we rely on the infrastructure of the kselftests framework so
that testing systems do not have to create "custom" frameworks to handle
all of the individual variants that could easily crop up here.
Let's keep it easy for people to run and use these tests, to not do so
is to ensure that they are not used, which is the exact opposite goal of
creating tests.
Greg,
It is easy for people (bpf developers) to run and use the tests.
Every developer runs them before submitting patches.
New tests is a hard requirement for any new features.
Maintainers run them for every push.
What I was and will push back hard is when other people (not bpf developers)
come back with an excuse that some CI system has a hard time running these
tests. It's the problem of weak CI. That CI needs to be fixed. Not the tests.
The example of this is that we already have github/libbpf CI that runs
selftests/bpf just fine. Anyone who wants to do another CI are welcome to copy
paste what already works instead of burdening people (bpf developers) who run
and use existing tests. I frankly have no sympathy to folks who put their own
interest of their CI development in front of bpf community of developers.
The main job of CI is to help developers and maintainers.
Where helping means to not impose new dumb rules on developers because CI
framework is dumb. Fix CI instead.
Here is what CI users are requesting:
- ability to install bpf test with other selftests using kselftest
install. The common framework is in place and with minor changes
to bpf test Makefile, we can make this happen. Others and myself
are willing to work on this, so we can get bpf test coverage in
test rings.
so you're saying that bpf maintainers and all bpf developers now
would need to incorporate new 'make install' step to their workflow
because some unknown CI system that is not even functional decided
to do 'make install' ?
That's exactly my point about selfish CI developers who put their
needs in front of bpf community of developers.
There is no need change bpf maintainer and developer workflow. You
don't have to use install option. Kselftest framework doesn't
require a specific workflow and you can:
1. Build and run your tests from bpf directory if you choose to
2. Install to run on different target.
Adding install install option requires a change to bpf Makefile
only to copy test that are built to install directory.
make kselftest-install from the main kernel Makefile in conjunction
with selftests Makefile and lib.mk will handle all of that.
Sounds like there is a misunderstanding that bpf maintainer/developer
workflow will have to change to support install. That is not the case.
The reason kselftest exists on the first place is to have common
framework to take care of build/run/install as a common layer so
individual test writers don't have to worry about these details
and write tests instead.
- be able to build and run existing tests without breaking the test
build when new tests are added that have hard dependency on new
versions of tools (llvm etc.). This isn't such a novel idea. We
don't break kernel builds every single release and even when we
require newer compiler releases. Plan the new tests with the intent
to not break existing users and add new tests at the same time.
We use min rev and not bleeding edge as the requirement for kernel
build.
'existing users'?
I said existing tests not users. When you add new bpf tests, existing
tests should continue to build and run without dependency on new revs
of llvm.
CI is not a user. CI is machine that should _help_
developers. Above two things (forcing install on humans and
breaking day-to-day tests for bpf maintainers and developers)
is the opposite of helping developers.
Please do NOT use such useless CI as an excuse.
Such CI should not be built in the first place when it slows down
the development instead of helping it.
LKFT test ring runs selftests now and Kernel CI will be soon. Various
users run selftests in their own environments. LKFT ring admins have
to build and install tests whether it is automated or not and look at
the results. If bpf test doesn't build and/or installed, it won't run
on test rings that qualify stable/next/main releases.
I don't understand why CI use-case is useless.
thanks,
-- Shuah
