Hi,
This is mass bug filing material, I have been under the weather
shortly after the first message on this thread but I have made more
progress now. Enough to CC the packaging list too.
On Fri, Oct 7, 2016 at 9:55 AM, Dominik 'Rathann' Mierzejewski
<dominik@xxxxxxxxxxxxxx> wrote:
> On Thursday, 06 October 2016 at 18:58, Dridi Boukelmoune wrote:
>> Hello,
>>
>> I was surprised to see /usr/share/texlive on my system although I
>> remembered very well removing it months ago. It turned out to be
>> caused by two rpmsave files, although some *empty* directories weren't
>> removed:
> [...]
> That's normal rpm behaviour. It only deletes directories if they're
> owned by the removed package and empty.
It was so obvious after you pointed it out that not figuring it out
myself felt quite embarrassing :)
>> But then I decided to check my whole /usr tree and found more orphans
>> than I expected.
>
> Feel free to open bugs against the offending packages. Complete file and
> directory ownership is mandated in the packaging guidelines.
Around the same time I had two ongoing reviews and realized that the
fedora-review tool warns about incomplete ownership inside built
packages, so I figured I could do something with DNF's filenames
databases via repoquery or something else.
However this approach has a high space and time complexity, because
for every single binary package it must check that all directories are
owned by the package or one of its dependencies with the exception of
directories owned by the filesystems package (and maybe others?)
So instead I wrote a proof of concept shell script that relies on installed
RPMs, because unless you purposefully install packages with missing
dependencies, a missing ownership of a directory _with owned contents_
is evidence of a packaging bug.
The script took about 30 minutes to run on my laptop, so I rewrote it in
bash to avoid forking and got below the 10 minutes mark. I then added
some cute reporting capabilities.
Example output for the /boot sub-tree on my daily driver:
grub2-efi-2.02-0.34.fc24.x86_64
└── /boot/efi/EFI/fedora
mactel-boot-0.9-13.fc24.x86_64
├── /boot/efi/.VolumeIcon.icns
├── /boot/efi/System/Library/CoreServices/SystemVersion.plist
├── /boot/efi/System/Library/CoreServices/boot.efi
└── /boot/efi/mach_kernel
shim-0.8-9.x86_64
├── /boot/efi/EFI/BOOT/BOOTX64.EFI
└── /boot/efi/EFI/BOOT/fallback.efi
For each package, the files that are listed have at least one parent
directory not owned.
Example's first culprits:
$ sudo rpm -qf /boot /boot/efi /boot/efi/EFI
filesystem-3.2-37.fc24.x86_64
file /boot/efi is not owned by any package
file /boot/efi/EFI is not owned by any package
For my RPMs sample, results are incomplete because:
- legacy pre-UsrMove paths are ignored
- symbolic links aren't followed
- no dependencies check
I did check dozens of results to refine the exceptions until I could no
longer find a false positive at random. The overall algorithm is dead
simple, so simple that I could implement it in bourne shell. It overlooks
many cases and probably has a high false-negative rate, but even
with such limitations the results are telling.
On my system, I found 2991 files "not fully owned" in 354 packages:
$ egrep '^\W' not-owned.txt | wc -l
2991
$ egrep '^\w' not-owned.txt | wc -l
354
I don't know how many directories are missing an owner.
The PoC can most likely be pushed further and even be comprehensive,
using a much much faster approach like DNF did for package resolution.
At this point it can only point out what's partially wrong, accurately.
The script is available [1] for anyone to try and review. It would work
best if people using different spins could try it, because I have less than
5000 packages installed.
Thoughts?
Cheers,
Dridi
PS. yes, I enjoy shell scripting, a lot
[1] https://dridi.fedorapeople.org/devel/not-fully-owned.sh
_______________________________________________
packaging mailing list -- packaging@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to packaging-leave@xxxxxxxxxxxxxxxxxxxxxxx
