On Mon, Aug 28, 2023 at 04:38:32PM +0200, Hans Verkuil wrote: > On 28/08/2023 16:26, Laurent Pinchart wrote: > > On Mon, Aug 28, 2023 at 04:14:56PM +0200, Hans Verkuil wrote: > >> On 28/08/2023 16:05, Jacopo Mondi wrote: > >>> On Mon, Aug 28, 2023 at 03:29:41PM +0200, Hans Verkuil wrote: > >>>> Hi all, > >>>> > >>>> We have been working on simplifying the media maintenance, and one part of that is > >>>> standardizing on build tools, in particular to make it easier for patch submitters > >>>> to run their patches through the same set of tests that the daily build does. > >>>> > >>>> This helps detect issues before you submit your patches. > >>>> > >>>> I have been working since July on transforming my hackish scripts to something > >>>> that is easier to use and of better quality. While there are still a few rough > >>>> edges, I consider it good enough to have others start to use it. > >>>> > >>>> To get the build scripts run: > >>>> > >>>> git clone git://linuxtv.org/hverkuil/build-scripts.git > >>>> > >>>> All the test builds will happen within this directory. It is completely separate > >>>> from where you do you normal development, instead you point it to where your > >>>> git repository is. > >>>> > >>>> See the README contained in the build-scripts git repo for all the details on > >>>> how to set it up. > >>>> > >>> > >>> I've been using your scripts since after ELC-E and I can tell they're > >>> useful! > >>> > >>>> Currently the scripts expect a debian 12-based distro (likely debian 11 will work > >>>> as well). I have no idea if it works well on Red Hat or Suse. If you use one of > >>>> those distros, and you get it to work, then a patch updating the README file with > >>>> the correct list of packages to install would be welcome. > >>> > >>> Speaking about distros, I was wondering if you still consider a > >>> requirement to build all compiler or we should instead try to use the > >>> distro provided ones when possible to test the distro-shipped version > >>> ? > >> > >> I strongly believe we should build the cross compilers. The reason is that > >> otherwise you get a wide variety of compiler versions, each with typically > >> different compiler warnings. It's a pain for a developer to see different > >> warnings than the person that merges those patches. > >> > >> It's a a regular problem that the daily build sees different warnings than > >> you do locally with the distro's compiler. > >> > >> Doing it this way also makes it easier to upgrade to the latest compiler > >> version, certainly quicker than a distro would do. > >> > >> It's about reproducibility, really. > > > > There's value in testing with different compiler versions though. The > > kernel's documented minimum gcc version is v5.1 at the moment. I > > certainly don't want to build myself with all versions between v5.1 and > > v13.2, but collectively we could cover more ground. > > > > Regardless of this, I already have recent cross compilers (built with > > buildroot) for ARM and ARM64, and I'd rather use those than duplicating > > compilers. Anything that consumes extra disk space is a serious > > hinderance. > > Feel free, but you run the risk that your PR is rejected because when I > run with these compiler versions I see new warnings. The whole point is > to be able to do the same tests before you make the PR to reduce the risk > of having to make a v2. > > FYI: the cross directory takes about 10 GB on my system. That can be > streamlined a bit by deleting some temporary directories needed while > building, probably to something closer to 5-6 GB. It may not be huge by itself, but it quickly adds up when you need to maintain multiple userspace cross-built enviroments, including Chrome OS, Android, Yocto, ... :-( I have half a TB of disk on my main development machine, and I would need at least 4 times that to cover my current needs comfortably. > >>>> Please note that running the regression tests using virtme-ng is currently only > >>>> supported on Debian 12, not on e.g. Ubuntu. Someone is looking into that, and > >>>> hopefully we can support that in the future. Running regressions tests are > >>>> primarily useful when making changes to core frameworks and public APIs, and > >>>> it is possible to run them manually (see the README). > >>>> > >>>> Since running this locally can take a fair amount of time, we hope to have > >>>> build servers available in the future so this can be offloaded. > >>>> > >>>> To give an idea of the expected build times: > >>>> > >>>> On an AMD Ryzen 9 6900HX (8 cores) a standard build of the staging tree > >>>> (build.sh -test all) takes 39 minutes. > >>>> > >>>> On an AMD Ryzen Threadripper 3970X (32 cores) it takes a bit over 13 minutes. -- Regards, Laurent Pinchart
