On 5/24/19 11:05 AM, Guillaume Tucker wrote: > On 24/05/2019 09:44, Hans Verkuil wrote: >> On 5/24/19 10:38 AM, Guillaume Tucker wrote: >>> Hi Hans, >>> >>> On 16/05/2019 07:41, Hans Verkuil wrote: >>>> Hi Guillaume, >>>> >>>> I have a few questions/suggestions: >>> >>> Thanks for the feedback! It's good to start seeing these reports >>> on the linux-media mailing list. And sorry for the slow reply, I >>> was away. >>> >>>> On 5/15/19 9:04 PM, kernelci.org bot wrote: >>>>> media/master v4l2-compliance on vivid: 236 tests, 0 regressions (media_v5.1-2-16-gfc8670d1f72b) >>>>> >>>>> Test results summary >>>>> -------------------- >>>>> >>>>> V4L2 Compliance on the vivid driver. >>>>> >>>>> This test ran "v4l2-compliance -s" from v4l-utils: >>>>> >>>>> https://www.linuxtv.org/wiki/index.php/V4l2-utils >>>> >>>> I'd just link directly to the git repo instead of the wiki: https://git.linuxtv.org/v4l-utils.git >>> >>> Sure, I thought this had been agreed before but it's easy to >>> change. >>> >>>> You should add the v4l-utils commit that's used to compile v4l2-compliance. >>>> That's important information to have. I assume that this test always uses the >>>> latest version of v4l-utils? >>> >>> This information is in the detailed results, but the detailed >>> results are only shown when there are some failures. So we'll >>> rework that a bit. >>> >>> For example, from the v4l2-compliance-uvc report: >>> >>> >>> Test failures >>> ------------- >>> 1 | rk3399-gru-kevin | arm64 | 52 total: 43 PASS 9 FAIL 0 SKIP >>> >>> Config: defconfig >>> Compiler: gcc-8 (aarch64-linux-gnu-gcc (Debian 8.3.0-2) 8.3.0) >>> Lab Name: lab-collabora >>> Plain log: https://storage.kernelci.org//media/master/media_v5.1-2-16-gfc8670d1f72b/arm64/defconfig/gcc-8/lab-collabora/v4l2-compliance-uvc-rk3399-gru-kevin.txt >>> HTML log: https://storage.kernelci.org//media/master/media_v5.1-2-16-gfc8670d1f72b/arm64/defconfig/gcc-8/lab-collabora/v4l2-compliance-uvc-rk3399-gru-kevin.html >>> Rootfs: http://storage.kernelci.org/images/rootfs/debian/stretch-v4l2/20190510.0/arm64/rootfs.cpio.gz >>> Test Git: git://linuxtv.org/v4l-utils.git >>> Test Commit: 0d61ddede7d340ffa1c75a2882e30c455ef3d8b8 >>> >>> >>> The git repo and commit hash here show you which version of >>> vl4-utils was used. >>> >>> >>> At the moment, the v4l2-compliance is part of a rootfs which gets >>> updated each time kernelci.org production code gets updated, >>> which is typically once a week. This can be improved to have the >>> rootfs updates independent from the rest, then we could trigger >>> rebuilds every time v4l-utils changes, but there are a few things >>> to take into consideration before we can do this safely. >> >> I don't think it has to be updated every time v4l-utils changes, at least >> for now, as long as it is clear which v4l-utils version is used. >> >>> >>>>> See each detailed section in the report below to find out the git URL and >>>>> particular revision that was used to build the test binaries. >>>>> >>>>> >>>>> Tree: media >>>>> Branch: master >>>>> Kernel: media_v5.1-2-16-gfc8670d1f72b >>>> >>>> I assume this is the version of the host kernel, right? Perhaps calling this >>>> "Host Kernel:" would be less ambiguous. >>> >>> I have to say I fail to see any ambiguity here: KernelCI is about >>> testing kernels, and this tells you the kernel revision under >>> test. Calling it "host" kernel might actually be confusing when >>> running with QEMU as people may think it's the version on the >>> host server running the test. >>> >>>>> URL: https://git.linuxtv.org/media_tree.git >>>>> Commit: fc8670d1f72b746ff3a5fe441f1fca4c4dba0e6f >> >> What confuses me is that the kernel above says: media_v5.1-2-16-gfc8670d1f72b >> but the commit says fc8670d1f72b746ff3a5fe441f1fca4c4dba0e6f. >> >> So the hashes are different, which led me to conclude that one referred to >> the host kernel and the other to the kernel under test. >> >> Where does the string "media_v5.1-2-16-gfc8670d1f72b" come from? > > This is the output of "git describe", except that the slash in > the tag name was replaced with an underscore (I know that's not > great, but otherwise it causes some issues with the path on the > storage server). The media/v5.2-1 tag has been created since > then, but if you run this: > > $ git checkout fc8670d1f72b746ff3a5fe441f1fca4c4dba0e6f > $ git tag -d media/v5.2-1 > $ git describe > media/v5.1-2-16-gfc8670d1f72b > > and fc8670d1f72b is the beginning of the full hash. Argh! I totally missed that. And even overlooked that 'g' is not a hex number :-) I never use git describe, so I wasn't familiar with the format, hence my confusion. So never mind my comment :-) Regards, Hans > >>>>> 1 | qemu | arm64 | 118 total: 118 PASS 0 FAIL 0 SKIP >>>>> 2 | qemu | arm | 118 total: 118 PASS 0 FAIL 0 SKIP >>>> >>>> Even if everything was OK, I think it would still be useful to have a link >>>> to the full test report. >>> >>> Yes, that is essentially the same issue as with the v4l-utils >>> version as I described above. The detailed results show a link >>> to the console output, which isn't just a clean v4l2-compliance >>> log but it's better than nothing. >>> >>> We may also add a feature to publish some files alongside the >>> parsed test results, and in the case of v4l2-compliance it would >>> typically be the plain output of the test suite that developers >>> are familiar with. It's not a supported feature right now as >>> only the raw console log is sent from the device to the database. >>> >>> Best wishes, >>> Guillaume >>> >> >> Regards, >> >> Hans >>
