will merge Rusticl tomorrow or so unless somebody complains. On Wed, Aug 24, 2022 at 5:34 PM Karol Herbst <kherbst@xxxxxxxxxx> wrote: > > On Wed, Aug 24, 2022 at 5:18 PM Jason Ekstrand > <jason.ekstrand@xxxxxxxxxxxxx> wrote: > > > > +mesa-dev and my jlekstrand.net e-mail > > > > On Sun, 2022-08-21 at 20:44 +0200, Karol Herbst wrote: > > > On Sun, Aug 21, 2022 at 8:34 PM Rob Clark <robdclark@xxxxxxxxx> > > > wrote: > > > > > > > > On Sun, Aug 21, 2022 at 10:45 AM Karol Herbst <kherbst@xxxxxxxxxx> > > > > wrote: > > > > > > > > > > On Sun, Aug 21, 2022 at 7:43 PM Karol Herbst <kherbst@xxxxxxxxxx> > > > > > wrote: > > > > > > > > > > > > On Sun, Aug 21, 2022 at 5:46 PM Rob Clark <robdclark@xxxxxxxxx> > > > > > > wrote: > > > > > > > > > > > > > > On Sat, Aug 20, 2022 at 5:23 AM Karol Herbst > > > > > > > <kherbst@xxxxxxxxxx> wrote: > > > > > > > > > > > > > > > > Hey everybody, > > > > > > > > > > > > > > > > so I think it's time to have this discussion for real. > > > > > > > > > > > > > > > > I am working on Rusticl > > > > > > > > ( > > > > > > > > https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15 > > > > > > > > 439) > > > > > > > > which I would like to merge quite soon. > > > > > > > > > > > > > > > > Others might also plan on starting kernel drivers written > > > > > > > > in Rust (and > > > > > > > > if people feel comfortable to discuss this as well, they > > > > > > > > might reply > > > > > > > > here) > > > > > > > > > > > > > > > > The overall implication of that is: if we are doing this, > > > > > > > > people (that > > > > > > > > is we) have to accept that touching Rust code will be part > > > > > > > > of our > > > > > > > > development process. There is no other sane way of doing > > > > > > > > it. > > > > > > > > > > > > > > > > I am not willing to wrap things in Rusticl so changing > > > > > > > > gallium APIs > > > > > > > > won't involve touching Rust code, and we also can't expect > > > > > > > > people to > > > > > > > > design their kernel drivers in weird ways "just because > > > > > > > > somebody > > > > > > > > doesn't want to deal with Rust" > > > > > > > > > > > > > > > > If we are going to do this, we have to do it for real, > > > > > > > > which means, > > > > > > > > Rust code will call C APIs directly and a change in those > > > > > > > > APIs will > > > > > > > > also require changes in Rust code making use of those APIs. > > > > > > > > > > > > > > > > I am so explicit on this very point, because we had some > > > > > > > > discussion on > > > > > > > > IRC where this was seen as a no-go at least from some > > > > > > > > people, which > > > > > > > > makes me think we have to find a mutual agreement on how it > > > > > > > > should be > > > > > > > > going forward. > > > > > > > > > > > > > > > > And I want to be very explicit here about the future of > > > > > > > > Rusticl as > > > > > > > > well: if the agreement is that people don't want to have to > > > > > > > > deal with > > > > > > > > Rust changing e.g. gallium, Rusticl is a dead project. I am > > > > > > > > not > > > > > > > > willing to come up with some trashy external-internal API > > > > > > > > just to > > > > > > > > maintain Rusticl outside of the mesa git repo. > > > > > > > > And doing it on a kernel level is even more of a no-go. > > > > > > > > > > > > > > > > So what are we all thinking about Rust in our core repos? > > > > > > > > > > > > > > I think there has to be willingness on the part of rust folks > > > > > > > to help > > > > > > > others who aren't so familiar with rust with these sorts of > > > > > > > API > > > > > > > changes. You can't completely impose the burden on others > > > > > > > who have > > > > > > > never touched rust before. That said, I expect a lot of API > > > > > > > changes > > > > > > > over time are simple enough that other devs could figure out > > > > > > > the > > > > > > > related rust side changes. > > > > > > > > > > > > > > > > > > > yeah, I agree here. I wouldn't say it's all the responsibility > > > > > > of > > > > > > developers changing APIs to also know how to change the code. > > > > > > So e.g. > > > > > > if an MR fails to compile and it's because of rusticl, I will > > > > > > help out > > > > > > and do the changes myself if necessary. But long term we have > > > > > > to > > > > > > accept that API changes also come with the implication of also > > > > > > having > > > > > > to touch Rust code. > > > > > > > > > > > > Short term it might be a learning opportunity for some/most, > > > > > > but long > > > > > > term it has to be accepted as a part of development to deal > > > > > > with Rust. > > > > > > > > > > > > > As long as folks who want to start introducing rust in mesa > > > > > > > and drm > > > > > > > realize they are also signing up to play the role of rust > > > > > > > tutor and > > > > > > > technical assistance, I don't see a problem. But if they > > > > > > > aren't > > > > > > > around and willing to help, I could see this going badly. > > > > > > > > > > > > > > > > > > > Yep, I fully agree here. This is also the main reason I am > > > > > > bringing > > > > > > this up. Nobody should be left alone with having to deal with > > > > > > changing > > > > > > the code. On the other hand a "not having to touch Rust code > > > > > > when > > > > > > changing APIs" guarantee is something which is simply > > > > > > impossible to > > > > > > have in any sane architecture. So we should figure out under > > > > > > which > > > > > > circumstances it will be okay for everybody. > > > > > > > > Yeah, this sounds fine to me. > > > > > > > > > > At least I don't see a way how I can structure Rusticl so that > > > > > > somebody working on gallium won't have to also deal with > > > > > > rusticl. One > > > > > > possibility would be to have a libgallium.so file I can link > > > > > > to, but > > > > > > then it's all about "stable gallium API" vs "not having to > > > > > > touch rust > > > > > > code" and I hope everybody prefers the second :) > > > > > > > > > > > > > > > > uhm... I meant "stable gallium API" vs "dealing with Rust code on > > > > > API changes" > > > > > > > > Yeah, stable gallium ABI makes as much sense as stable drm (in- > > > > kernel) > > > > ABI (ie. it doesn't). So even if we tried to make it so that folks > > > > who haven't used rust much (or at all) yet on the gallium side, > > > > we'll > > > > still have to deal with it on the kernel side. So let's not design > > > > our APIs to avoid a learning curve.. lets' just recognize that the > > > > learning curve will be there. > > > > I think this is mostly fine. If you're making a gallium change that > > touches all the drivers, you need cross-project review as well. > > Learning a bit of rust or asking one of the rust-knowledgable people to > > update rusticl's wrappers seems reasonable at that point. > > > > Where I would take issue is if various wrappers or inlines are > > duplicated in rust such that a change which wouldn't break drivers > > because they're all using the wrappers do break rust code. I know > > that, at least at one point, rusticl had some of this which is why I'm > > pointing it out specifically. I've not looked at the branch in a > > while, though. > > > > The only thing I am aware of is that I have an iterator written in > rust for glsl's list implementation in order to iterate over variables > inside nir shaders. But yeah, that has the potential to blow up if > somebody decides to change how the list works without changing the > definition of the list. But as I am actually running piglit on rusticl > with llvmpipe, this might not be a problem we wouldn't catch. > > > > > > > > I do also wonder a bit about code tooling (indexers, etc).. > > > > > > > I'm not > > > > > > > sure what the state of things when it comes to cross c<->rust > > > > > > > integration. Ie. it is usually straightforward enough to > > > > > > > track down > > > > > > > all the spots in C code which would be affected by some > > > > > > > change. It > > > > > > > might be easier to overlook things on the rust side. On the > > > > > > > mesa > > > > > > > side, pre-merge CI jobs help to catch these issues. Less > > > > > > > sure about > > > > > > > how to handle that on the kernel side. > > > > > > > > > > > > > > > > > > > At least for Rusticl it's all within meson/ninja. We use > > > > > > bindgen to > > > > > > generate the bindings automatically so you simply run into > > > > > > compilation > > > > > > issues. And for the kernel side I think that Linus wanted Rust > > > > > > to be > > > > > > non optional. If something uses it, you also make sure the Rust > > > > > > side > > > > > > compiles. And the build system is dynamic enough that you can > > > > > > also > > > > > > wire up bindgen and make it part of the normal development > > > > > > model. > > > > > > > > > > > > In regards to code tooling, for rust you usually rely on > > > > > > rust-analyzer. I've already figured out with dcbaker on how to > > > > > > integrate our meson setup with that. Atm I am able to get the > > > > > > full > > > > > > experience with VScode. Not sure if we also need to figure out > > > > > > how > > > > > > that can work with e.g. vim and how to deal with C <=> Rust > > > > > > boundaries. > > > > > > > > Yeah, I've used vscode with rust codebase (crosvm).. just not for a > > > > project that includes the combination of a significant amount of > > > > both > > > > .c and .rs. > > > > > > > > > > we have meson generate a rust-project.json file which you can point > > > rust-analyzer to: https://github.com/mesonbuild/meson/pull/10682 > > > > > > Sadly there is no "real" integration of rust and C code, so all you > > > get are references to the bindgen generated files, but you can't jump > > > to the actual C headers from there. > > > > I would be very surprised if any sort of C refactoring tool could see > > through bindgen now, or in the future. Seeing across language > > boundaries is tractable in C because it's basically a superset of C++. > > (Yes, I know it's not really.) I wouldn't expect a static analyzer to > > see across a C/Python or C/Java boundary, so I don't expect it to be > > able to see across C/rust. (Sadly, the boundary is sharp enough that > > that's a reasonable comparison.) > > > > > > If bindgen is integrated with the build system, that sounds fine as > > > > far as helping to not overlook changes that are needed on the .rs > > > > side > > > > of things.. crosvm seems to mainly make bindgen a manual step, but > > > > then again it is mostly used for external dependencies like the > > > > interface to virglrenderer > > > > > > > > Side note, from my experience with crosvm, please be careful about > > > > any > > > > external rust dependencies that are used.. rust still seems to be > > > > young enough and evolving quickly enough that thing can quickly > > > > turn > > > > into a big headache > > > > > > > > > > That's easy: meson doesn't support external crates, so no external > > > rust dependencies are used at the moment :) > > > > > > But yeah, I was planning on using some where it really makes sense, > > > but we can have this discussion once we get there. So far it all > > > works > > > without any external dependency besides bindgen. > > > > I think most of us are agreed here. Mesa has pretty strict rules about > > pulling in piles of external dependencies and we don't want that to > > change just because we added a bit of rust. There may be a crate or > > two that we want to pull in in the future but, for now, everything is > > core. > > > > One point not mentioned above is Rust compiler versions. Because Rust > > is a rapidly evolving language, new features are being added to the > > language and standard library every day. If we impose the same > > requirements on Rust that we're imposing on GCC where we still compile > > with a version from 10-15 years ago, that's going to really hamper > > development. We probably don't want to require people to always pull > > the very latest compiler from nightly but I do think we want to be able > > to bump the compiler version once a year or something. IDK how distros > > and others will feel about that. Hopefully, they feel about the same > > way towards compiler requirement bumps as they feel towards rust > > itself. > > > > yeah.. atm I actually require 1.59 which is quite new (Feb 2022). I > think we might want to make sure that at least debians rust compiles > rusticl in the future, but they also seem to be willing to packaging > specific versions if they need it for other things like Firefox. They > have rustc (1.48) and rustc-mozilla (1.51), where the latter is even > available in stretch. So we might actually decide to simply depend on > the same rust version Firefox uses, because that's something > distributions will have to deal with anyway. > > > --Jason > > > > > > BR, > > > > -R > > > > > > > > > > > BR, > > > > > > > -R > > > > > > > > > > > > > > > CCing a bunch of people who think are the most impacted by > > > > > > > > it either > > > > > > > > from a reviewer/maintainer or developer perspective. If you > > > > > > > > think some > > > > > > > > others should be explicitly aware of this, please point > > > > > > > > them to this > > > > > > > > discussion here. > > > > > > > > > > > > > > > > Karol > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
