On 24/05/2019 10:20, Hans Verkuil wrote: > On 5/24/19 11:05 AM, Guillaume Tucker wrote: >> On 24/05/2019 09:44, Hans Verkuil wrote: >>> On 5/24/19 10:38 AM, Guillaume Tucker wrote: >>>> Hi Hans, >>>> >>>> On 16/05/2019 07:41, Hans Verkuil wrote: >>>>> Hi Guillaume, >>>>> >>>>> I have a few questions/suggestions: >>>> >>>> Thanks for the feedback! It's good to start seeing these reports >>>> on the linux-media mailing list. And sorry for the slow reply, I >>>> was away. >>>> >>>>> On 5/15/19 9:04 PM, kernelci.org bot wrote: >>>>>> media/master v4l2-compliance on vivid: 236 tests, 0 regressions (media_v5.1-2-16-gfc8670d1f72b) >>>>>> >>>>>> Test results summary >>>>>> -------------------- >>>>>> >>>>>> V4L2 Compliance on the vivid driver. >>>>>> >>>>>> This test ran "v4l2-compliance -s" from v4l-utils: >>>>>> >>>>>> https://www.linuxtv.org/wiki/index.php/V4l2-utils >>>>> >>>>> I'd just link directly to the git repo instead of the wiki: https://git.linuxtv.org/v4l-utils.git >>>> >>>> Sure, I thought this had been agreed before but it's easy to >>>> change. >>>> >>>>> You should add the v4l-utils commit that's used to compile v4l2-compliance. >>>>> That's important information to have. I assume that this test always uses the >>>>> latest version of v4l-utils? >>>> >>>> This information is in the detailed results, but the detailed >>>> results are only shown when there are some failures. So we'll >>>> rework that a bit. >>>> >>>> For example, from the v4l2-compliance-uvc report: >>>> >>>> >>>> Test failures >>>> ------------- >>>> 1 | rk3399-gru-kevin | arm64 | 52 total: 43 PASS 9 FAIL 0 SKIP >>>> >>>> Config: defconfig >>>> Compiler: gcc-8 (aarch64-linux-gnu-gcc (Debian 8.3.0-2) 8.3.0) >>>> Lab Name: lab-collabora >>>> Plain log: https://storage.kernelci.org//media/master/media_v5.1-2-16-gfc8670d1f72b/arm64/defconfig/gcc-8/lab-collabora/v4l2-compliance-uvc-rk3399-gru-kevin.txt >>>> HTML log: https://storage.kernelci.org//media/master/media_v5.1-2-16-gfc8670d1f72b/arm64/defconfig/gcc-8/lab-collabora/v4l2-compliance-uvc-rk3399-gru-kevin.html >>>> Rootfs: http://storage.kernelci.org/images/rootfs/debian/stretch-v4l2/20190510.0/arm64/rootfs.cpio.gz >>>> Test Git: git://linuxtv.org/v4l-utils.git >>>> Test Commit: 0d61ddede7d340ffa1c75a2882e30c455ef3d8b8 >>>> >>>> >>>> The git repo and commit hash here show you which version of >>>> vl4-utils was used. >>>> >>>> >>>> At the moment, the v4l2-compliance is part of a rootfs which gets >>>> updated each time kernelci.org production code gets updated, >>>> which is typically once a week. This can be improved to have the >>>> rootfs updates independent from the rest, then we could trigger >>>> rebuilds every time v4l-utils changes, but there are a few things >>>> to take into consideration before we can do this safely. >>> >>> I don't think it has to be updated every time v4l-utils changes, at least >>> for now, as long as it is clear which v4l-utils version is used. >>> >>>> >>>>>> See each detailed section in the report below to find out the git URL and >>>>>> particular revision that was used to build the test binaries. >>>>>> >>>>>> >>>>>> Tree: media >>>>>> Branch: master >>>>>> Kernel: media_v5.1-2-16-gfc8670d1f72b >>>>> >>>>> I assume this is the version of the host kernel, right? Perhaps calling this >>>>> "Host Kernel:" would be less ambiguous. >>>> >>>> I have to say I fail to see any ambiguity here: KernelCI is about >>>> testing kernels, and this tells you the kernel revision under >>>> test. Calling it "host" kernel might actually be confusing when >>>> running with QEMU as people may think it's the version on the >>>> host server running the test. >>>> >>>>>> URL: https://git.linuxtv.org/media_tree.git >>>>>> Commit: fc8670d1f72b746ff3a5fe441f1fca4c4dba0e6f >>> >>> What confuses me is that the kernel above says: media_v5.1-2-16-gfc8670d1f72b >>> but the commit says fc8670d1f72b746ff3a5fe441f1fca4c4dba0e6f. >>> >>> So the hashes are different, which led me to conclude that one referred to >>> the host kernel and the other to the kernel under test. >>> >>> Where does the string "media_v5.1-2-16-gfc8670d1f72b" come from? >> >> This is the output of "git describe", except that the slash in >> the tag name was replaced with an underscore (I know that's not >> great, but otherwise it causes some issues with the path on the >> storage server). The media/v5.2-1 tag has been created since >> then, but if you run this: >> >> $ git checkout fc8670d1f72b746ff3a5fe441f1fca4c4dba0e6f >> $ git tag -d media/v5.2-1 >> $ git describe >> media/v5.1-2-16-gfc8670d1f72b >> >> and fc8670d1f72b is the beginning of the full hash. > > Argh! I totally missed that. And even overlooked that 'g' is not a hex > number :-) > > I never use git describe, so I wasn't familiar with the format, hence my > confusion. > > So never mind my comment :-) No worries, and yeah the hashes are not in heptadecimal :) Best wishes, Guillaume > Regards, > > Hans > >> >>>>>> 1 | qemu | arm64 | 118 total: 118 PASS 0 FAIL 0 SKIP >>>>>> 2 | qemu | arm | 118 total: 118 PASS 0 FAIL 0 SKIP >>>>> >>>>> Even if everything was OK, I think it would still be useful to have a link >>>>> to the full test report. >>>> >>>> Yes, that is essentially the same issue as with the v4l-utils >>>> version as I described above. The detailed results show a link >>>> to the console output, which isn't just a clean v4l2-compliance >>>> log but it's better than nothing. >>>> >>>> We may also add a feature to publish some files alongside the >>>> parsed test results, and in the case of v4l2-compliance it would >>>> typically be the plain output of the test suite that developers >>>> are familiar with. It's not a supported feature right now as >>>> only the raw console log is sent from the device to the database. >>>> >>>> Best wishes, >>>> Guillaume >>>> >>> >>> Regards, >>> >>> Hans >>> >
