Re: [libgpiod][PATCH 0/2] bindings: rust: feature gate unreleased features

[Bartosz Golaszewski <brgl@xxxxxxxx>]

On Mon, Oct 9, 2023 at 1:38 PM Erik Schilling <erik.schilling@xxxxxxxxxx> wrote:
>
> On Mon Oct 9, 2023 at 10:58 AM CEST, Bartosz Golaszewski wrote:
> > On Fri, Oct 6, 2023 at 9:24 AM Erik Schilling <erik.schilling@xxxxxxxxxx> wrote:
> > >
> > > When releasing the 0.2.0 version of the libgpiod crate, I did not
> > > realize that there were C lib features that were not released yet.
> > > Helpfully, vhost-device's CI went up in flames and revealed this [1].
> > >
> > > This suggests a way to handle that and sketches how further updates can
> > > be handled.
> > >
> > > I acknowledge that this may be very strange to C developers...
> > > Traditionally, one would just use whatever your distro provided and the
> > > distro would make sure that dependencies update in lock-step.
> > >
> > > However, in Rust the default way to consume libraries is to pull them
> > > from crates.io. This is a balancing act for -sys crates which link to
> > > distro-provided libraries on the system. Since crates.io does not wait
> > > for distros to update their libraries, crates will need to support a
> > > wider range of system libraries.
> > >
> > > This sets up / sketches the infrastructure for that.
> > >
> > > Only the first commit is intended to be merged. The second one just
> > > sketches how a release will look like once it happens.
> > >
> > > [1] https://buildkite.com/rust-vmm/vhost-device-ci/builds/1746#018b0110-b9d3-468a-973c-c3bbc27cd479
> > >
> > > To: Bartosz Golaszewski <brgl@xxxxxxxx>
> > > To: Linux-GPIO <linux-gpio@xxxxxxxxxxxxxxx>
> > > Cc: Viresh Kumar <viresh.kumar@xxxxxxxxxx>
> > > Cc: Manos Pitsidianakis <manos.pitsidianakis@xxxxxxxxxx>
> > > Cc: Kent Gibson <warthog618@xxxxxxxxx>
> > >
> > > Signed-off-by: Erik Schilling <erik.schilling@xxxxxxxxxx>
> > > ---
> > > Erik Schilling (2):
> > >       bindings: rust: feature gate unreleased features
> > >       DONOTMERGE: bindings: rust: simulate v2.1 release
> > >
> > >  bindings/rust/libgpiod-sys/Cargo.toml        |  9 +++++++--
> > >  bindings/rust/libgpiod/Cargo.toml            |  4 ++++
> > >  bindings/rust/libgpiod/Makefile.am           |  2 +-
> > >  bindings/rust/libgpiod/README.md             | 14 ++++++++++++++
> > >  bindings/rust/libgpiod/src/line_request.rs   |  2 ++
> > >  bindings/rust/libgpiod/tests/line_request.rs |  1 +
> > >  6 files changed, 29 insertions(+), 3 deletions(-)
> > > ---
> > > base-commit: e7b02c2259d97c77107c77b68e3bc1664e6703c1
> > > change-id: 20231006-b4-bindings-old-version-fix-789973703b77
> > >
> > > Best regards,
> > > --
> > > Erik Schilling <erik.schilling@xxxxxxxxxx>
> > >
> >
> > I'm currently discussing a similar problem with a developer who
> > offered to help make Python bindings released on PyPi more
> > user-friendly. His suggestion for python bindings is what many PyPi
> > packages that provide bindings to C libraries do: just compile the
> > library statically and make it part of the C extension module bundled
> > together with python code.
> >
> > Given that rust programs are statically linked, maybe this is a
> > solution we could use?
>
> It may be one option that we could provide, but I still see need for
> linking against system libs.
>

Sure, this is what the python bindings will get. A switch to link
against the system libgpiod if desired.

> Rust is already a bit special - at least compared to traditional C
> projects - in that it pulls all the code of all dependencies and builds
> the entire bunch into a static lib. This is still upsetting some people
> who prefer each dependency to be its own self-contained dynamic library
> that can be updated without updating the rest of it. Rust can make
> that work by being a mono-culture in term of build tools. Everything on
> crates.io builds through cargo in the same way, so tooling can assist
> with auditing and updating crates.
>
> While distros - to various degrees - are starting to accept that this
> may be a viable approach, I imagine that they would still feel uneasy
> about some Rust code statically linking a C lib. They spend a lot of
> effort making sure that the C lib is built according to their packaging
> guidelines and do not want to repeat that effort for each Rust binary
> that ends up including libgpiod.
>
> Hence, I think that we would still need to provide a way to build and
> link against the system libs.
>
> However, that does not necessarily means
> that we need to feature gate pending features like suggested here.
> We _could_ just tie the Rust bindings to the libgpiod version 1:1 and
> make bindings only support one version of the C lib.
>
> That would come with some implications for the release  though. We would
> effectively need to manage bugfixes to the bindings through libgpiod's
> stable branches. For the version numbers to be non-confusing, we would
> probably also need to always do the releases together. So there may
> be needs to do a new patch release for libgpiod even if only a fix was
> needed in the bindings.
>
> Then, we may end up in a situation where SemVer changes may be
> conflicting between bindings and the core lib. With the Rust bindings
> being new, my recent fix of a soundness issue required changes to types
> that required a bump of the major version. The C version was entirely
> unaffected. Should we have bumped its major for this too?
>

Absolutely not! I expect rust bindings to still be quite volatile in
terms of API stability but libgpiod v2 is pretty mature. Distros
already seem to be slow adopting the new release so I definitely don't
want to confuse them even more by bumping libgpiod major version
because of a change in rust bindings.

We decided to fully decouple rust from libgpiod and let's keep it that way.

> I outlined these kind of issues in my initial thread around publishing
> to crates.io [1]. We could of course just document somewhere which
> version of the Rust crate pulls in which version of the C lib.
>
> But is it worth it? We would still need to maintain the nested build
> next to building against system libs. libgpiod is of course fairly
> boring to build. But we would still need to tell users to install the
> few build dependencies, C tools may update and the latest crates.io
> release may fail building.

The core libgpiod library requires a sane toolchain and standard
library with GNU extensions period. That will not change and I'm quite
convinced that if an environment has cargo and rustcc and it will have
a C compiler as well.

Bart

>
> I totally see that static builds may be helpful in some cases. But I
> would assume that the maintenance overhead of nesting the build is less
> controllable compared to what I suggest in this patch. The C lib already
> follows the good practices around versioning and API compatibility. So
> I would assume that maintaining the conditional compiles will be fairly
> simple.
>
> Once another major bump happens in the C lib, we can just follow the
> bump by removing all existing conditionals and bumping the required
> floor version for the Rust bindings.
>
> All in all, I only see minor advantages for users while seeing a lot of
> maintenance burden.
>
> [1] https://lore.kernel.org/all/CT0CRWOTJIEO.20BGIDMLFM0E8@fedora/
>
> - Erik
>