On Wed, Jun 12, 2024 at 6:20 PM Laurent Pinchart <laurent.pinchart@xxxxxxxxxxxxxxxx> wrote: > > On Wed, Jun 12, 2024 at 06:01:07PM +0900, Tomasz Figa wrote: > > On Wed, Jun 12, 2024 at 5:34 PM Laurent Pinchart wrote: > > > On Wed, Jun 12, 2024 at 05:22:34PM +0900, Tomasz Figa wrote: > > > > On Wed, Jun 12, 2024 at 4:54 PM Mauro Carvalho Chehab wrote: > > > > > Em Wed, 12 Jun 2024 08:46:50 +0200 Hans Verkuil escreveu: > > > > > > On 6/12/24 06:12, Tomasz Figa wrote: > > > > > > > On Wed, May 15, 2024 at 1:19 AM Daniel Almeida wrote: > > > > > > >> > > > > > > >> Hi Hans, all, > > > > > > >> > > > > > > >> I’d like to attend in person and discuss the use of Rust in the subsystem, especially in light of [0] and [1]. > > > > > > >> > > > > > > >> Please note that these are new submissions that are unrelated with what was discussed last year. > > > > > > >> > > > > > > >> 30 minutes will do. > > > > > > >> > > > > > > >> [0] https://lwn.net/ml/linux-media/20240227215146.46487-1-daniel.almeida@xxxxxxxxxxxxx/ > > > > > > >> [1] https://lwn.net/Articles/970565 > > > > > > > > > > > > > > Somewhat related to the topic: I see potential for a quite big > > > > > > > redesign of the videobuf2 framework going forward and recently with > > > > > > > more Rust adoption I'm starting to think it could benefit from being > > > > > > > implemented in Rust, since we would have to rewrite it quite a bit > > > > > > > anyway. Especially since it's a part of the subsystem that has to deal > > > > > > > with memory management, object lifetime and asynchronousness quite a > > > > > > > lot and we had a history of issues there. So it could be interesting > > > > > > > to hear everyone's thoughts. > > > > > > > > > > > > I think it is far too soon to write a framework like that in Rust. > > > > > > > > > > Agreed. I don't object redesigns in C to make it better - which could have > > > > > some colateral effect of making things easier for a future Rust adoption, > > > > > but such changes should be justified by themselves, and not because of a > > > > > language change. > > > > > > > > No, the thought of redesign doesn't come from the language change, > > > > it's the other way around. Since rewriting a lot of the code already, > > > > why not do it in a language that is generally considered better. > > > > > > > > > See: redesigns at the core will potentially affect lots of drivers, > > > > > so it needs very good technical reasons why doing it. Plus, it requires > > > > > comprehensive tests with different types of hardware/drivers to reduce the > > > > > risk of regressions. Depending on the changes, it may require extra tests > > > > > with devices that are outside complex camera world: radio, analog and digital > > > > > TV drivers - and even some input devices that use VB2 - to ensure that > > > > > nothing broke. > > > > > > > > We don't have to do it in an all-or-nothing way. We can start with an > > > > experimental new implementation in Rust, which could be gradually > > > > tested. It could even be done the same way as the vb -> vb2 > > > > transition, although I suspect it wouldn't really be necessary, as I > > > > would like to see it more like a drop-in replacement. In general I > > > > think the API exposed outside of the framework wouldn't really change > > > > that much, it's more about the internal design. > > > > > > > > > > To be > > > > > > honest, I won't even consider it until Linus officially accepts Rust as a > > > > > > second language in the kernel, instead of as an experiment. > > > > > > > > > > This is not enough: if the core starts to use a second language, all media > > > > > developers will be affected and will be required to have expertise on such > > > > > language. > > > > > > > > Let's be realistic, how many developers are actively touching vb2 these days? > > > > > > > > > That's not something that should happen without careful > > > > > analysis and plans that should include a gradual roll-up, lost of tests > > > > > with the affected drivers including the legacy ones and some strategy to > > > > > quickly solve regression issues. > > > > > > > > That said, I agree. It needs proper discussion and planning. That's > > > > why I'm proposing this as a topic. :) > > > > Moreover the redesign itself also needs proper discussion and is more > > > > of a long term goal, not something to land in the next few days. > > > > > > Focussing on this topic, if we're brainstorming memory management for > > > media devices, I'd like to throw in a controversial idea. In addition to > > > being clearer on the fact that USERPTR is deprecated, > > > > Definitely. This has been long overdue. > > > > > I would like to > > > deprecate MMAP too and only focus on DMABUF. I believe Linux needs a > > > centralized buffer allocator, instead of having multiple allocation APIs > > > scattered in different places. There are design ideas in gralloc that we > > > could benefit from. > > > > Given that we now have DMA-buf heaps in mainline, it sounds much more > > realistic. It would certainly help eliminate some issues in the vb2 > > layer, such as vb2-dma-sg having its own open coded page allocation > > that can't handle DMA addressing limitations (which can be fixed with > > some additions to the DMA API, but eliminating the problem altogether > > is way better than any other solution.) > > There are (at least) two issues we'll have to solve to make DMA heaps > more universally usable: > > - Memory allocation from DMA heaps isn't accounted in the process > resource limits. This is one of the blockers for getting distributions > to enable the heaps. We'll have to fix that. I think the same applies to MMAP buffers today or am I missing something? That said, it's something that has to be fixed anyway. DMA heaps would make it easier to fix, since we would have one central place where the accounting could be implemented. > > - We need a userspace library to pick the right heap based on the memory > constraints of the devices that you'll want to share the buffer with. > This will require exposing those constraints to userspace somehow. That is not a new problem either. Today with MMAP we also need the userspace to somehow choose the right exporter. So we could start with each video node pointing to the right heap (probably for each plane separately?), which would provide the same level of functionality as with MMAP today, but using a uniform API and a central implementation in the kernel. Then we could follow up with exposing more information, if needed, to help the userspace make a more informed decision. > > I'm sure there will be more issues, but solving issues is what we do :-) > > > That said, as we already use a centralized DMA-buf allocator in > > ChromeOS and don't really care about the MMAP mode, I'm definitely > > biased here. We would need to hear from people working on userspace > > which still uses it (if there is any). > > > > > > > It is not a matter of what language is better. Instead, it is a matter of > > > > > not affecting code maintenance during the (probably long) transition period > > > > > and beyond. > > > > > > > > > > If you see the past history, the transition from V4L to V4L2 took more than 10 > > > > > years - being possible to be done only with the help of libv4l, plus a > > > > > lot of backward-compat code that we added. Still there were several > > > > > regressions and we even had to quickly patch the Kernel and/or some apps > > > > > that were using the uAPI on different ways. > > > > > > > > That's a different situation, because UAPI is involved. > > > > > > > > > Yet, the transition from VB1 to VB2 was also painful, and took a lot of time. > > > > > > > > Yes, vb -> vb2 would be a more appropriate comparison. > > > > > > > > > On both cases, there were very good technical reasons for the transition, > > > > > in terms of missing needed features, broken memory models and serious > > > > > troubles that utterly causing VB1 to not work well on non-x86 hardware. > > > > > > > > It's a very similar situation now, vb2 doesn't work well on modern > > > > hardware, but I still have hopes that it can be fixed without > > > > affecting the driver-facing behavior. (We would probably need to > > > > develop some unit tests that validate the driver-facing behavior to > > > > ensure that.) > > > > > > > > > In the end, the authors of the core changes need to acquire legacy hardware > > > > > and to do lots of driver-specific changes to avoid breaking existing stuff. > > > > > Hans and I had to dedicate a lot of time and efforts on such transitions, > > > > > as it required a lot of work. > > > > > > > > > > I can tell you: there's no fun on such changes: typically, companies won't > > > > > pay someone to do changes on drivers for legacy hardware, specially > > > > > when there are no real benefits, which is the case here, as the final result > > > > > is just to keep the existing drivers to work with existing hardware, > > > > > usually without any new features. So, the ones behind such core changes > > > > > have to commit fixing drivers usually on their spare time. > > > > > > > > I don't get that argument. Wouldn't the same apply to any core change? > > > > I think the reason we have driver maintainers is that they can help > > > > with testing. Moreover, we need to invest into testing infrastructure > > > > (which is what people have been doing recently via Media CI) to make > > > > such changes less painful. Otherwise the subsystem will just bit-rot > > > > and become useful for modern use cases. > > > > > > I've recently seen an increase in people experimenting with sourdough, > > > kombucha, kimchi and other fermentation techniques, so rotting isn't > > > always negative [*], but I assume you meant useless here, not useful :-) > > > > Yeah, definitely. I'd love it if bit-rotting led to computer software > > becoming more useful, but sadly it's rarely the case. > > > > > * I'll draw the line at surströmming. > > > > > > > > > The vb2 framework can certainly use some more work, and esp. better support > > > > > > for codecs, since that's where the main pain is at the moment. > > > > > > > > > > > > But I would need to see a proper proposal first. I assume that's what you > > > > > > plan to present? > > > > > > > > > > > > > That said, I wouldn't be able to travel this time unfortunately, so it > > > > > > > would be nice if we could arrange this topic in a time slot friendly > > > > > > > for remote attendance from Japan. Also +Hidenori Kobayashi from my > > > > > > > team who would also be interested in joining remotely. > > > > > > > > > > > > That would mean a slot in the morning, right? Since Japan is 7 hours ahead > > > > > > of CEST. > > -- > Regards, > > Laurent Pinchart
