Re: [libgpiod][PATCH 0/2] bindings: rust: feature gate unreleased features

["Erik Schilling" <erik.schilling@xxxxxxxxxx>]

On Mon Oct 9, 2023 at 10:58 AM CEST, Bartosz Golaszewski wrote:
> On Fri, Oct 6, 2023 at 9:24 AM Erik Schilling <erik.schilling@xxxxxxxxxx> wrote:
> >
> > When releasing the 0.2.0 version of the libgpiod crate, I did not
> > realize that there were C lib features that were not released yet.
> > Helpfully, vhost-device's CI went up in flames and revealed this [1].
> >
> > This suggests a way to handle that and sketches how further updates can
> > be handled.
> >
> > I acknowledge that this may be very strange to C developers...
> > Traditionally, one would just use whatever your distro provided and the
> > distro would make sure that dependencies update in lock-step.
> >
> > However, in Rust the default way to consume libraries is to pull them
> > from crates.io. This is a balancing act for -sys crates which link to
> > distro-provided libraries on the system. Since crates.io does not wait
> > for distros to update their libraries, crates will need to support a
> > wider range of system libraries.
> >
> > This sets up / sketches the infrastructure for that.
> >
> > Only the first commit is intended to be merged. The second one just
> > sketches how a release will look like once it happens.
> >
> > [1] https://buildkite.com/rust-vmm/vhost-device-ci/builds/1746#018b0110-b9d3-468a-973c-c3bbc27cd479
> >
> > To: Bartosz Golaszewski <brgl@xxxxxxxx>
> > To: Linux-GPIO <linux-gpio@xxxxxxxxxxxxxxx>
> > Cc: Viresh Kumar <viresh.kumar@xxxxxxxxxx>
> > Cc: Manos Pitsidianakis <manos.pitsidianakis@xxxxxxxxxx>
> > Cc: Kent Gibson <warthog618@xxxxxxxxx>
> >
> > Signed-off-by: Erik Schilling <erik.schilling@xxxxxxxxxx>
> > ---
> > Erik Schilling (2):
> >       bindings: rust: feature gate unreleased features
> >       DONOTMERGE: bindings: rust: simulate v2.1 release
> >
> >  bindings/rust/libgpiod-sys/Cargo.toml        |  9 +++++++--
> >  bindings/rust/libgpiod/Cargo.toml            |  4 ++++
> >  bindings/rust/libgpiod/Makefile.am           |  2 +-
> >  bindings/rust/libgpiod/README.md             | 14 ++++++++++++++
> >  bindings/rust/libgpiod/src/line_request.rs   |  2 ++
> >  bindings/rust/libgpiod/tests/line_request.rs |  1 +
> >  6 files changed, 29 insertions(+), 3 deletions(-)
> > ---
> > base-commit: e7b02c2259d97c77107c77b68e3bc1664e6703c1
> > change-id: 20231006-b4-bindings-old-version-fix-789973703b77
> >
> > Best regards,
> > --
> > Erik Schilling <erik.schilling@xxxxxxxxxx>
> >
>
> I'm currently discussing a similar problem with a developer who
> offered to help make Python bindings released on PyPi more
> user-friendly. His suggestion for python bindings is what many PyPi
> packages that provide bindings to C libraries do: just compile the
> library statically and make it part of the C extension module bundled
> together with python code.
>
> Given that rust programs are statically linked, maybe this is a
> solution we could use?

It may be one option that we could provide, but I still see need for
linking against system libs.

Rust is already a bit special - at least compared to traditional C
projects - in that it pulls all the code of all dependencies and builds
the entire bunch into a static lib. This is still upsetting some people
who prefer each dependency to be its own self-contained dynamic library
that can be updated without updating the rest of it. Rust can make
that work by being a mono-culture in term of build tools. Everything on
crates.io builds through cargo in the same way, so tooling can assist
with auditing and updating crates.

While distros - to various degrees - are starting to accept that this
may be a viable approach, I imagine that they would still feel uneasy
about some Rust code statically linking a C lib. They spend a lot of
effort making sure that the C lib is built according to their packaging
guidelines and do not want to repeat that effort for each Rust binary
that ends up including libgpiod.

Hence, I think that we would still need to provide a way to build and
link against the system libs.

However, that does not necessarily means
that we need to feature gate pending features like suggested here.
We _could_ just tie the Rust bindings to the libgpiod version 1:1 and
make bindings only support one version of the C lib.

That would come with some implications for the release  though. We would
effectively need to manage bugfixes to the bindings through libgpiod's
stable branches. For the version numbers to be non-confusing, we would
probably also need to always do the releases together. So there may
be needs to do a new patch release for libgpiod even if only a fix was
needed in the bindings.

Then, we may end up in a situation where SemVer changes may be
conflicting between bindings and the core lib. With the Rust bindings
being new, my recent fix of a soundness issue required changes to types
that required a bump of the major version. The C version was entirely
unaffected. Should we have bumped its major for this too?

I outlined these kind of issues in my initial thread around publishing
to crates.io [1]. We could of course just document somewhere which
version of the Rust crate pulls in which version of the C lib.

But is it worth it? We would still need to maintain the nested build
next to building against system libs. libgpiod is of course fairly
boring to build. But we would still need to tell users to install the
few build dependencies, C tools may update and the latest crates.io
release may fail building.

I totally see that static builds may be helpful in some cases. But I
would assume that the maintenance overhead of nesting the build is less
controllable compared to what I suggest in this patch. The C lib already
follows the good practices around versioning and API compatibility. So
I would assume that maintaining the conditional compiles will be fairly
simple.

Once another major bump happens in the C lib, we can just follow the
bump by removing all existing conditionals and bumping the required
floor version for the Rust bindings.

All in all, I only see minor advantages for users while seeing a lot of
maintenance burden.

[1] https://lore.kernel.org/all/CT0CRWOTJIEO.20BGIDMLFM0E8@fedora/

- Erik