Hi Josef, Sorry about the late reply; I messed up my domain last week and emails were getting black holed. On Tue, Feb 28, 2023 at 10:38:18AM -0500, Josef Bacik wrote: > On Mon, Feb 27, 2023 at 6:02 PM Daniel Xu <dxu@xxxxxxxxx> wrote: > > > > === Introduction === > > > > Testing is paradoxically one of BPF's great strengths as well as one of it's > > current weaknesses. Fortunately, this weakness is not too far from being > > corrected. > > > > BPF_PROG_RUN is somewhat of a double edged sword. On the one hand, you can run > > reproducibly run progs in near-production context. On the other hand, since BPF > > is so deeply intertwined with the running kernel, you must make the kernel you > > run tests on as close to your production kernel as possible to get full testing > > benefits. > > > > This is going to be more of an issue going forward through the growth of kfuncs > > because kfuncs do not possess a stable ABI [5]. Proper testing should be > > encouraged at a community-wide level in order to avoid accidental surprises and > > potential loss of faith in BPF "stability". > > > > Most successful kernel-dependent projects deploy some form of > > virtual-machine-based testing [1][2][3][4] to solve the above issues. However, > > there are two problems with this: > > > > 1. VM-based testing is not quite common knowledge yet and remains somewhat of > > a dark art to successfully implement. > > > > 2. Multiple implementations of what is essentially the same thing is somewhat > > of a drain on resources. > > > > (These are not necessarily bad things -- it is useful and necessary to explore a > > problem space before settling on best practices) > > > > vmtest [0] aims to solve both problems. > > > > === Goals === > > > > I'd like to do a short presentation on the design and ideas behind vmtest. I'd > > also like to show a quick demo. It shouldn't take very long. I'll probably > > also share what I'd like to implement next. I don't know what that's going > > to be at time of writing b/c I'm probably going to get to it before LSFMMBPF. > > > > For the rest of the time I'd like to discuss what the community would like to > > see in vmtest. And to hear what it'd take to see adoption from other projects. > > Obviously no one can be required to adopt vmtest but I think it'll save everyone > > a good deal of effort if done correctly. > > > > FYI a lot of us have been working on kdevops > (https://github.com/linux-kdevops/kdevops), which has similar goals, > tho yours feels more in line with virt-me which I've used a bunch as > well. > > I would very much love it if we could all get behind one project. The > benefit of kdevops is it's very extensible, and being able to select > some config options and have an entire testing suite up and running is > very handy for new developers. Thanks for the tip; I had not heard of kdevops. I'll spend some time this week playing with it. > That being said I can definitely get behind have two sort of options, > the bigger swiss army knife that is kdevops, and something smaller > that's easier to do one-off runs. > > kdevops using vagrant makes a lot of the pain of setting up the full > VM environments that existed before go away. I can easily tear down > my 8 CI vm's and rebuild them all and having them testing again in 4 > commands. Nowadays I probably wouldn't use virt-me/anything lighter > because this is actually pretty easy to get up and running. > > Again no trying to be discouraging, but you're absolutely right that > there's been a lot of fragmentation here, which is why I've spent > probably a lot more time than I should have making kdevops work for > me, as well have a lot of other kernel developers, and it's getting > pretty solid. Thanks, Yep, ideally I'd like to have pulled something off the shelf and moved on. But nothing looked like it would solve my problem. So if kdevops is close enough then I wouldn't mind contributing to that instead. Thanks, Daniel
