On 3/19/25 1:55 AM, David Gibson wrote:
>> diff --git a/MANIFEST.in b/MANIFEST.in
>> deleted file mode 100644
>> index 904d124..0000000
>> --- a/MANIFEST.in
>> +++ /dev/null
>> @@ -1,12 +0,0 @@
>> -# SPDX-License-Identifier: (GPL-2.0-or-later OR BSD-2-Clause)
>> -
>> -global-exclude *
>> -include README.md
>> -include GPL
>> -include BSD-2-Clause
>> -include setup.py
>> -include pylibfdt/libfdt.i
>> -include libfdt/libfdt.h
>> -include libfdt/fdt.h
>> -include libfdt/libfdt_env.h
>> -include VERSION.txt
>
> We can't just delete the MANIFEST.in and setup.py, for now, because
> they're still needed for the Makefile build. I'm certainly
> considering dropping support for make, but we're not there yet.
If, as discussed in the previous emails, we drop support for *building
python from Make* but don't drop Make for C, is that acceptable?
>> diff --git a/libfdt/meson.build b/libfdt/meson.build
>> index c2f4bd6..9c6ef54 100644
>> --- a/libfdt/meson.build
>> +++ b/libfdt/meson.build
>> @@ -31,7 +31,7 @@ libfdt = library(
>> version: meson.project_version(),
>> link_args: link_args,
>> link_depends: 'version.lds',
>> - install: true,
>> + install: get_option('default_library') != 'static' or not wheel_only,
>
> The first clause doesn't quite make sense to me. Why would we not
> install a static library?
Read this the other way around.
We *do* install, if either (or both) conditions are true:
- we are building a non-static library, so we certainly need it at
runtime (thus it has to be installed at runtime)
- we are NOT building a special wheel-specific build, which is to say,
we assume we need this installed if we aren't running from pip install
i.e. the purpose of this is that we assume the end goal for python
wheels is as I'm going to describe in the latter half of this email, and
this specific conditional over here is just trying to detect "PyPI mode".
>> +[tool.meson-python.args]
>> +'setup' = ['-Ddefault_library=static', '-Dwheel-only=true']
>
> Shouldn't the python library use the shared library, at least by
> default?
This isn't about building the python library. If I'm building the python
library, I just use "meson setup" like normal, and it builds the full
dtc stack, including tools, library, python bindings and all.
meson-python is *specifically* about building a python wheel. Python
wheels have two choices:
- the shared library is copied into the wheel
- don't use the shared library
Without setting these options, a wheel uploaded to PyPI ends up looking
like this:
$ bsdtar -tf dist/libfdt-1.7.2-cp312-cp312-linux_x86_64.whl
libfdt-1.7.2.dist-info/METADATA
libfdt-1.7.2.dist-info/WHEEL
.libfdt.mesonpy.libs/libfdt.so.1.7.2
.libfdt.mesonpy.libs/pkgconfig/libfdt.pc
libfdt-1.7.2.data/scripts/convert-dtsv0
libfdt-1.7.2.data/scripts/dtc
libfdt-1.7.2.data/scripts/fdtdump
libfdt-1.7.2.data/scripts/fdtget
libfdt-1.7.2.data/scripts/fdtput
libfdt-1.7.2.data/scripts/fdtoverlay
libfdt-1.7.2.data/scripts/dtdiff
libfdt.py
_libfdt.cpython-312-x86_64-linux-gnu.so
libfdt-1.7.2.data/headers/fdt.h
libfdt-1.7.2.data/headers/libfdt.h
libfdt-1.7.2.data/headers/libfdt_env.h
libfdt-1.7.2.dist-info/RECORD
And,
$ readelf -d _libfdt.cpython-312-x86_64-linux-gnu.so
Library runpath: [$ORIGIN/.libfdt.mesonpy.libs]
This is all necessary since pip install can't know that libfdt is
already installed via a package manager (dpkg, rpm, portage, pacman,
xbps, apk, etc), so it has to ship its own just in case.
Building statically simplifies distribution as a PyPI wheel.
If you really want to build bloated wheels, you can still do it:
python -m build --wheel \
-Csetup-args=-Ddefault_library=static \
-Csetup-args=wheel-only=false
By the way, the locations that the files get installed to when building
via pip aren't great either. The *.data/headers/ gets installed to
/usr/include/python3.12/libfdt/
and the library and pkg-config files end up in
/usr/lib/python3.12/site-packages/.libfdt.mesonpy.libs/libfdt.so.1.7.2
/usr/lib/python3.12/site-packages/.libfdt.mesonpy.libs/pkgconfig/libfdt.pc
The pkg-config file doesn't even get remapped, it still says
-I/usr/include and -L/usr/lib64
If you install with pip install --user (default when not running with
sudo), then you even end up with this:
/home/eschwartz/.local/bin/convert-dtsv0
/home/eschwartz/.local/bin/dtc
/home/eschwartz/.local/bin/dtdiff
/home/eschwartz/.local/bin/fdtdump
/home/eschwartz/.local/bin/fdtget
/home/eschwartz/.local/bin/fdtoverlay
/home/eschwartz/.local/bin/fdtput
/home/eschwartz/.local/include/python3.12/libfdt/fdt.h
/home/eschwartz/.local/include/python3.12/libfdt/libfdt.h
/home/eschwartz/.local/include/python3.12/libfdt/libfdt_env.h
/home/eschwartz/.local/lib/python3.12/site-packages/.libfdt.mesonpy.libs/libfdt.so.1.7.2
/home/eschwartz/.local/lib/python3.12/site-packages/.libfdt.mesonpy.libs/pkgconfig/libfdt.pc
/home/eschwartz/.local/lib/python3.12/site-packages/_libfdt.cpython-312-x86_64-linux-gnu.so
/home/eschwartz/.local/lib/python3.12/site-packages/libfdt.py
So yeah, IMO all of this cruft is out of place for pypi.org / pip.
Python packaging *cannot*, in any way shape or form be used outside of
its closed-in silo. They literally forbid, via their standards body, the
possibility of installing outside of the locations I described. The only
way to install to arbitrary locations the way Make or Meson can, is to
use what they describe as "the legacy install route" of running
```
python setup.py install
```
and then having the equivalent of Makefile
`cp XXXX ${DESTDIR}${PREFIX}/bin`
i.e. completely disrespecting the pip install approach.
The point of distributing on PyPI is so that people can get a
python-specific copy of the bindings easily, with a minimum of fuss, and
nothing else. You can always get the complete set using a real build
system and installing with "meson install" instead of pip, and usually
using a distro package manager even. But people installing via pip
install aren't necessarily interested in that. They just want to
download a binary package which their python scripts depend on.
--
Eli Schwartz
Attachment:
OpenPGP_signature.asc
Description: OpenPGP digital signature
