On 13/06/2024 10:14, Tomasz Figa wrote: > On Thu, Jun 13, 2024 at 4:38 PM Hans Verkuil <hverkuil@xxxxxxxxx> wrote: >> >> On 12/06/2024 22:35, Mauro Carvalho Chehab wrote: >>> Em Wed, 12 Jun 2024 17:22:34 +0900 >>> Tomasz Figa <tfiga@xxxxxxxxxxxx> escreveu: >>> >>>> On Wed, Jun 12, 2024 at 4:54 PM Mauro Carvalho Chehab >>>> <mchehab@xxxxxxxxxx> wrote: >>>>> >>>>> Em Wed, 12 Jun 2024 08:46:50 +0200 >>>>> Hans Verkuil <hverkuil@xxxxxxxxx> escreveu: >>>>> >>>>>> On 6/12/24 06:12, Tomasz Figa wrote: >>>>>>> On Wed, May 15, 2024 at 1:19 AM Daniel Almeida >>>>>>> <daniel.almeida@xxxxxxxxxxxxx> wrote: >>>>>>>> >>>>>>>> Hi Hans, all, >>>>>>>> >>>>>>>> I’d like to attend in person and discuss the use of Rust in the subsystem, especially in light of [0] and [1]. >>>>>>>> >>>>>>>> Please note that these are new submissions that are unrelated with what was discussed last year. >>>>>>>> >>>>>>>> 30 minutes will do. >>>>>>>> >>>>>>>> [0] https://lwn.net/ml/linux-media/20240227215146.46487-1-daniel.almeida@xxxxxxxxxxxxx/ >>>>>>>> [1] https://lwn.net/Articles/970565 >>>>>>> >>>>>>> Somewhat related to the topic: I see potential for a quite big >>>>>>> redesign of the videobuf2 framework going forward and recently with >>>>>>> more Rust adoption I'm starting to think it could benefit from being >>>>>>> implemented in Rust, since we would have to rewrite it quite a bit >>>>>>> anyway. Especially since it's a part of the subsystem that has to deal >>>>>>> with memory management, object lifetime and asynchronousness quite a >>>>>>> lot and we had a history of issues there. So it could be interesting >>>>>>> to hear everyone's thoughts. >>>>>> >>>>>> I think it is far too soon to write a framework like that in Rust. >>>>> >>>>> Agreed. I don't object redesigns in C to make it better - which could have >>>>> some colateral effect of making things easier for a future Rust adoption, >>>>> but such changes should be justified by themselves, and not because of a >>>>> language change. >>>> >>>> No, the thought of redesign doesn't come from the language change, >>>> it's the other way around. Since rewriting a lot of the code already, >>>> why not do it in a language that is generally considered better. >>> >>> As Hans said, Rast has experimental support. We can't have drivers >>> depending on experimental stuff. >> >> Indeed. >> >> While discussing Rust for experimental drivers or codec libraries is >> interesting (and I am doing a Rust course, so hopefully I have a better >> understanding of what's involved by the upcoming media summit), using >> it for core media frameworks is simply a hard NACK until Linus blesses >> Rust as a second kernel language. >> >> So don't spend your valuable time on that. >> > > Alright. I'm fine with C as well, although it's a shame that > eventually when Rust becomes a first-class citizen we'll be left with > a lot of legacy code base. Anyway, I guess let's wait until that > happens first. :) > >>> >>>> >>>>> >>>>> See: redesigns at the core will potentially affect lots of drivers, >>>>> so it needs very good technical reasons why doing it. Plus, it requires >>>>> comprehensive tests with different types of hardware/drivers to reduce the >>>>> risk of regressions. Depending on the changes, it may require extra tests >>>>> with devices that are outside complex camera world: radio, analog and digital >>>>> TV drivers - and even some input devices that use VB2 - to ensure that >>>>> nothing broke. >>>> >>>> We don't have to do it in an all-or-nothing way. We can start with an >>>> experimental new implementation in Rust, which could be gradually >>>> tested. It could even be done the same way as the vb -> vb2 >>>> transition, although I suspect it wouldn't really be necessary, as I >>>> would like to see it more like a drop-in replacement. In general I >>>> think the API exposed outside of the framework wouldn't really change >>>> that much, it's more about the internal design. >> >> It makes no sense to have a C and a Rust version of vb2. This framework >> is critical to all drivers, and we're not going to support two versions >> and fix bugs/add features in two places. Again, it's a hard NACK. Don't >> waste time on that. >> >> If there are ideas to make vb2 better, then I am all for that. >> >> I just want to mention two things here: >> >> For most drivers, using vb2 is just fine: the work a driver needs to do is >> quite straightforward. Exceptions are codec drivers and possibly complex >> camera drivers when they need to use requests (not certain yet). > > Do we have any data that suggests that non-codec and non-complex > camera drivers are actually "most drivers"? Anything with a simple video pipeline: webcams and PCI/USB video capture cards in particular. All devices you probably never work with since they are primary PC oriented and not for embedded systems/SoCs. I have drawers full of them... Codec drivers are a minority. > > Anyway, I agree that "using" vb2 is indeed fine and I want us to keep > using it. But whether it actually works well is a different story. > Things become problematic as soon as someone intends to run something > more complex than yavta, e.g. exporting and importing DMA-bufs is > involved. > > So putting aside the Rust discussion (that wasn't really the core > point), could I get 30 minutes to cover the vb2 pain points and how we > could fix them? Or should we just work on that and send patches? > Either works for me. (In fact we started already, e.g. via the > duplicate plane mapping patch series). Sure! Can you make a new reply to this topic with a proposal? This is hidden inside this longer thread, so I'd rather have a fresh post. Regards, Hans > >> >> Internally vb2 is quite complex, but that's because what it does is quite >> complex. And that's fine. If the internal structure can be improved to >> make it less complex, then I'm all for that, but there is no magic bullet >> (including using Rust instead of C) that suddenly makes everything simple. >> >> Generally I prefer to have the complexity in core frameworks, that will >> only make life easier for the driver developers. > > I prefer complexity neither in drivers nor core frameworks, but we > can't have everything. ;) > > I agree that buffer management is a complex problem, so we can't avoid > some level of complexity, although there is certainly room for > improvement in vb2. That also wasn't the core reason for the proposed > redesign. The core point is about the functional issues. > >> >> To summarize: >> >> Until Rust is accepted by Linus as a second kernel language, as media >> maintainer I will NACK core media frameworks written in Rust. I won't >> spend time on it, it's an immediate NACK from me. >> >> Note that this doesn't imply that once Linus *does* accept Rust, that we >> are OK with core frameworks written in Rust. But that will be a separate >> discussion once that happens. > > Ack. > > Best regards, > Tomasz > >> >> Regards, >> >> Hans >> >>> >>> Having two implementations of the same logic doesn't sound reasonable, >>> as it doubles the maintainership effort: all changes done on one >>> implementation needs to be moved to the other one. >>> >>> Btw, we also have seem this problem before with VB and, up to some >>> sense, with VB2, as some drivers used to have their own buffer >>> handling implementation that usually started from a VB or VB2 fork. >>> >>> So, if VB2 has issues, let's fix it in C code. >>> >>>>>> To be >>>>>> honest, I won't even consider it until Linus officially accepts Rust as a >>>>>> second language in the kernel, instead of as an experiment. >>>>> >>>>> This is not enough: if the core starts to use a second language, all media >>>>> developers will be affected and will be required to have expertise on such >>>>> language. >>>> >>>> Let's be realistic, how many developers are actively touching vb2 these days? >>> >>> How many developers don't need VB2? Hopefully none :-) >>> >>>>> That's not something that should happen without careful >>>>> analysis and plans that should include a gradual roll-up, lost of tests >>>>> with the affected drivers including the legacy ones and some strategy to >>>>> quickly solve regression issues. >>>> >>>> That said, I agree. It needs proper discussion and planning. That's >>>> why I'm proposing this as a topic. :) >>>> Moreover the redesign itself also needs proper discussion and is more >>>> of a long term goal, not something to land in the next few days. >>>> >>>>> >>>>> It is not a matter of what language is better. Instead, it is a matter of >>>>> not affecting code maintenance during the (probably long) transition period >>>>> and beyond. >>>>> >>>>> If you see the past history, the transition from V4L to V4L2 took more than 10 >>>>> years - being possible to be done only with the help of libv4l, plus a >>>>> lot of backward-compat code that we added. Still there were several >>>>> regressions and we even had to quickly patch the Kernel and/or some apps >>>>> that were using the uAPI on different ways. >>>> >>>> That's a different situation, because UAPI is involved. >>> >>> It is different, but similar, up to some sense, as a change at VB2 >>> implementation will likely affect its kAPI, its behavior or both. >>> >>> The point I'm underlining is that core redesigns do affect existing >>> drivers usually on unexpected ways. >>> >>>> >>>>> >>>>> Yet, the transition from VB1 to VB2 was also painful, and took a lot of time. >>>>> >>>> >>>> Yes, vb -> vb2 would be a more appropriate comparison. >>>> >>>>> On both cases, there were very good technical reasons for the transition, >>>>> in terms of missing needed features, broken memory models and serious >>>>> troubles that utterly causing VB1 to not work well on non-x86 hardware. >>>>> >>>> >>>> It's a very similar situation now, vb2 doesn't work well on modern >>>> hardware, but I still have hopes that it can be fixed without >>>> affecting the driver-facing behavior. (We would probably need to >>>> develop some unit tests that validate the driver-facing behavior to >>>> ensure that.) >>>> >>>>> In the end, the authors of the core changes need to acquire legacy hardware >>>>> and to do lots of driver-specific changes to avoid breaking existing stuff. >>>>> Hans and I had to dedicate a lot of time and efforts on such transitions, >>>>> as it required a lot of work. >>>>> >>>>> I can tell you: there's no fun on such changes: typically, companies won't >>>>> pay someone to do changes on drivers for legacy hardware, specially >>>>> when there are no real benefits, which is the case here, as the final result >>>>> is just to keep the existing drivers to work with existing hardware, >>>>> usually without any new features. So, the ones behind such core changes >>>>> have to commit fixing drivers usually on their spare time. >>>>> >>>> >>>> I don't get that argument. Wouldn't the same apply to any core change? >>> >>> It depends of the type of change. For instance, an addition of a new >>> V4L2 control should not cause regressions to existing drivers. The >>> same would be true if one adds a new memory allocation component for >>> VB2 (e. g. something similar to videobuf2-vmalloc.c/videobuf2-dma-sg.c/..): >>> only drivers using the new way would be affected. >>> >>>> I think the reason we have driver maintainers is that they can help >>>> with testing. Moreover, we need to invest into testing infrastructure >>>> (which is what people have been doing recently via Media CI) to make >>>> such changes less painful. Otherwise the subsystem will just bit-rot >>>> and become useful for modern use cases. >>> >>> Using CI to check for uAPI/kAPI changes is helpful, but it doesn't cover >>> actual drivers. For that, we would need to invest on a CI solution >>> integrated with lots of different hardware pieces, to check for actual >>> driver regressions. >>> >>> On one of my previous work, the company I used to work had that: they >>> had some monitors display some things, and the camera captured input >>> were compared to what the monitor were actually displaying. Doable, but >>> expensive. >>> >>> Regards, >>> Mauro >>> >>> Thanks, >>> Mauro >> >
