On Wed, Jun 12, 2024 at 06:33:14PM +0900, Tomasz Figa wrote: > On Wed, Jun 12, 2024 at 6:20 PM Laurent Pinchart wrote: > > On Wed, Jun 12, 2024 at 06:01:07PM +0900, Tomasz Figa wrote: > > > On Wed, Jun 12, 2024 at 5:34 PM Laurent Pinchart wrote: > > > > On Wed, Jun 12, 2024 at 05:22:34PM +0900, Tomasz Figa wrote: > > > > > On Wed, Jun 12, 2024 at 4:54 PM Mauro Carvalho Chehab wrote: > > > > > > Em Wed, 12 Jun 2024 08:46:50 +0200 Hans Verkuil escreveu: > > > > > > > On 6/12/24 06:12, Tomasz Figa wrote: > > > > > > > > On Wed, May 15, 2024 at 1:19 AM Daniel Almeida wrote: > > > > > > > >> > > > > > > > >> Hi Hans, all, > > > > > > > >> > > > > > > > >> I’d like to attend in person and discuss the use of Rust in the subsystem, especially in light of [0] and [1]. > > > > > > > >> > > > > > > > >> Please note that these are new submissions that are unrelated with what was discussed last year. > > > > > > > >> > > > > > > > >> 30 minutes will do. > > > > > > > >> > > > > > > > >> [0] https://lwn.net/ml/linux-media/20240227215146.46487-1-daniel.almeida@xxxxxxxxxxxxx/ > > > > > > > >> [1] https://lwn.net/Articles/970565 > > > > > > > > > > > > > > > > Somewhat related to the topic: I see potential for a quite big > > > > > > > > redesign of the videobuf2 framework going forward and recently with > > > > > > > > more Rust adoption I'm starting to think it could benefit from being > > > > > > > > implemented in Rust, since we would have to rewrite it quite a bit > > > > > > > > anyway. Especially since it's a part of the subsystem that has to deal > > > > > > > > with memory management, object lifetime and asynchronousness quite a > > > > > > > > lot and we had a history of issues there. So it could be interesting > > > > > > > > to hear everyone's thoughts. > > > > > > > > > > > > > > I think it is far too soon to write a framework like that in Rust. > > > > > > > > > > > > Agreed. I don't object redesigns in C to make it better - which could have > > > > > > some colateral effect of making things easier for a future Rust adoption, > > > > > > but such changes should be justified by themselves, and not because of a > > > > > > language change. > > > > > > > > > > No, the thought of redesign doesn't come from the language change, > > > > > it's the other way around. Since rewriting a lot of the code already, > > > > > why not do it in a language that is generally considered better. > > > > > > > > > > > See: redesigns at the core will potentially affect lots of drivers, > > > > > > so it needs very good technical reasons why doing it. Plus, it requires > > > > > > comprehensive tests with different types of hardware/drivers to reduce the > > > > > > risk of regressions. Depending on the changes, it may require extra tests > > > > > > with devices that are outside complex camera world: radio, analog and digital > > > > > > TV drivers - and even some input devices that use VB2 - to ensure that > > > > > > nothing broke. > > > > > > > > > > We don't have to do it in an all-or-nothing way. We can start with an > > > > > experimental new implementation in Rust, which could be gradually > > > > > tested. It could even be done the same way as the vb -> vb2 > > > > > transition, although I suspect it wouldn't really be necessary, as I > > > > > would like to see it more like a drop-in replacement. In general I > > > > > think the API exposed outside of the framework wouldn't really change > > > > > that much, it's more about the internal design. > > > > > > > > > > > > To be > > > > > > > honest, I won't even consider it until Linus officially accepts Rust as a > > > > > > > second language in the kernel, instead of as an experiment. > > > > > > > > > > > > This is not enough: if the core starts to use a second language, all media > > > > > > developers will be affected and will be required to have expertise on such > > > > > > language. > > > > > > > > > > Let's be realistic, how many developers are actively touching vb2 these days? > > > > > > > > > > > That's not something that should happen without careful > > > > > > analysis and plans that should include a gradual roll-up, lost of tests > > > > > > with the affected drivers including the legacy ones and some strategy to > > > > > > quickly solve regression issues. > > > > > > > > > > That said, I agree. It needs proper discussion and planning. That's > > > > > why I'm proposing this as a topic. :) > > > > > Moreover the redesign itself also needs proper discussion and is more > > > > > of a long term goal, not something to land in the next few days. > > > > > > > > Focussing on this topic, if we're brainstorming memory management for > > > > media devices, I'd like to throw in a controversial idea. In addition to > > > > being clearer on the fact that USERPTR is deprecated, > > > > > > Definitely. This has been long overdue. > > > > > > > I would like to > > > > deprecate MMAP too and only focus on DMABUF. I believe Linux needs a > > > > centralized buffer allocator, instead of having multiple allocation APIs > > > > scattered in different places. There are design ideas in gralloc that we > > > > could benefit from. > > > > > > Given that we now have DMA-buf heaps in mainline, it sounds much more > > > realistic. It would certainly help eliminate some issues in the vb2 > > > layer, such as vb2-dma-sg having its own open coded page allocation > > > that can't handle DMA addressing limitations (which can be fixed with > > > some additions to the DMA API, but eliminating the problem altogether > > > is way better than any other solution.) > > > > There are (at least) two issues we'll have to solve to make DMA heaps > > more universally usable: > > > > - Memory allocation from DMA heaps isn't accounted in the process > > resource limits. This is one of the blockers for getting distributions > > to enable the heaps. We'll have to fix that. > > I think the same applies to MMAP buffers today or am I missing something? Correct. We could fix that too, but I'd rather focus on DMA heaps. > That said, it's something that has to be fixed anyway. DMA heaps would > make it easier to fix, since we would have one central place where the > accounting could be implemented. > > > - We need a userspace library to pick the right heap based on the memory > > constraints of the devices that you'll want to share the buffer with. > > This will require exposing those constraints to userspace somehow. > > That is not a new problem either. Today with MMAP we also need the > userspace to somehow choose the right exporter. So we could start with > each video node pointing to the right heap (probably for each plane > separately?), which would provide the same level of functionality as > with MMAP today, but using a uniform API and a central implementation > in the kernel. > > Then we could follow up with exposing more information, if needed, to > help the userspace make a more informed decision. The two issues can be decoupled for the implementation phase, true, but I think we'll have to showcase a design that solves them both if we want to get adoption fo this API shift. > > I'm sure there will be more issues, but solving issues is what we do :-) > > > > > That said, as we already use a centralized DMA-buf allocator in > > > ChromeOS and don't really care about the MMAP mode, I'm definitely > > > biased here. We would need to hear from people working on userspace > > > which still uses it (if there is any). > > > > > > > > > It is not a matter of what language is better. Instead, it is a matter of > > > > > > not affecting code maintenance during the (probably long) transition period > > > > > > and beyond. > > > > > > > > > > > > If you see the past history, the transition from V4L to V4L2 took more than 10 > > > > > > years - being possible to be done only with the help of libv4l, plus a > > > > > > lot of backward-compat code that we added. Still there were several > > > > > > regressions and we even had to quickly patch the Kernel and/or some apps > > > > > > that were using the uAPI on different ways. > > > > > > > > > > That's a different situation, because UAPI is involved. > > > > > > > > > > > Yet, the transition from VB1 to VB2 was also painful, and took a lot of time. > > > > > > > > > > Yes, vb -> vb2 would be a more appropriate comparison. > > > > > > > > > > > On both cases, there were very good technical reasons for the transition, > > > > > > in terms of missing needed features, broken memory models and serious > > > > > > troubles that utterly causing VB1 to not work well on non-x86 hardware. > > > > > > > > > > It's a very similar situation now, vb2 doesn't work well on modern > > > > > hardware, but I still have hopes that it can be fixed without > > > > > affecting the driver-facing behavior. (We would probably need to > > > > > develop some unit tests that validate the driver-facing behavior to > > > > > ensure that.) > > > > > > > > > > > In the end, the authors of the core changes need to acquire legacy hardware > > > > > > and to do lots of driver-specific changes to avoid breaking existing stuff. > > > > > > Hans and I had to dedicate a lot of time and efforts on such transitions, > > > > > > as it required a lot of work. > > > > > > > > > > > > I can tell you: there's no fun on such changes: typically, companies won't > > > > > > pay someone to do changes on drivers for legacy hardware, specially > > > > > > when there are no real benefits, which is the case here, as the final result > > > > > > is just to keep the existing drivers to work with existing hardware, > > > > > > usually without any new features. So, the ones behind such core changes > > > > > > have to commit fixing drivers usually on their spare time. > > > > > > > > > > I don't get that argument. Wouldn't the same apply to any core change? > > > > > I think the reason we have driver maintainers is that they can help > > > > > with testing. Moreover, we need to invest into testing infrastructure > > > > > (which is what people have been doing recently via Media CI) to make > > > > > such changes less painful. Otherwise the subsystem will just bit-rot > > > > > and become useful for modern use cases. > > > > > > > > I've recently seen an increase in people experimenting with sourdough, > > > > kombucha, kimchi and other fermentation techniques, so rotting isn't > > > > always negative [*], but I assume you meant useless here, not useful :-) > > > > > > Yeah, definitely. I'd love it if bit-rotting led to computer software > > > becoming more useful, but sadly it's rarely the case. > > > > > > > * I'll draw the line at surströmming. > > > > > > > > > > > The vb2 framework can certainly use some more work, and esp. better support > > > > > > > for codecs, since that's where the main pain is at the moment. > > > > > > > > > > > > > > But I would need to see a proper proposal first. I assume that's what you > > > > > > > plan to present? > > > > > > > > > > > > > > > That said, I wouldn't be able to travel this time unfortunately, so it > > > > > > > > would be nice if we could arrange this topic in a time slot friendly > > > > > > > > for remote attendance from Japan. Also +Hidenori Kobayashi from my > > > > > > > > team who would also be interested in joining remotely. > > > > > > > > > > > > > > That would mean a slot in the morning, right? Since Japan is 7 hours ahead > > > > > > > of CEST. -- Regards, Laurent Pinchart
