Re: No space left errors on shutdown with systemd-homed /home dir

[Chris Murphy <lists@xxxxxxxxxxxxxxxxx>]

On Wed, Jan 26, 2022 at 4:19 PM Boris Burkov <boris@xxxxxx> wrote:
>
> On Thu, Jan 27, 2022 at 12:07:53AM +0200, Apostolos B. wrote:
> >  This is what homectl inspect user reports:
> >
> >   Disk Size: 128.0G
> >   Disk Usage: 3.8G (= 3.1%)
> >   Disk Free: 124.0G (= 96.9%)
> >
> > and this is what btrfs usage reports:
> >
> > sudo btrfs filesystem usage /home/toliz
> >
> > Overall:
> >     Device size:             127.98GiB
> >     Device allocated:               4.02GiB
> >     Device unallocated:     123.96GiB
> >     Device missing:                 0.00B
> >     Used:                           1.89GiB
> >     Free (estimated):             124.10GiB    (min: 62.12GiB)
> >     Free (statfs, df):             124.10GiB
> >     Data ratio:                  1.00
> >     Metadata ratio:                  2.00
> >     Global reserve:               5.14MiB    (used: 0.00B)
> >     Multiple profiles:                    no
> >
> > Data,single: Size:2.01GiB, Used:1.86GiB (92.73%)
> >    /dev/mapper/home-toliz       2.01GiB
> >
> > Metadata,DUP: Size:1.00GiB, Used:12.47MiB (1.22%)
> >    /dev/mapper/home-toliz       2.00GiB
> >
> > System,DUP: Size:8.00MiB, Used:16.00KiB (0.20%)
> >    /dev/mapper/home-toliz      16.00MiB
> >
> > Unallocated:
> >    /dev/mapper/home-toliz     123.96GiB
> >
>
> OK, there is plenty of unallocated space, thanks for confirming.
>
> Looking at the stack trace a bit more, the only thing that really sticks
> out as suspicious to me is btrfs_shrink_device, I'm not sure who would
> want to do that or why.

systemd-homed by default uses btrfs on LUKS on loop mount, with a
backing file. On login, it grows the user home file system with some
percentage (I think 80%) of the free space of the underlying file
system. And on logout, it does both fstrim and shrinks the fs. I don't
know why it does both, it seems adequate to do only fstrim on logout
to return unused blocks to the underlying file system; and to do an fs
resize on login to either grow or shrink the user home file system.

But also, we don't really have a great estimator of the minimum size a
file system can be. `btrfs inspect-internal min-dev-size` is pretty
broken right now.
https://github.com/kdave/btrfs-progs/issues/271

I'm not sure if systemd folks would use libbtrfsutil facility to
determine the minimum device shrink size? But also even the kernel
doesn't have a very good idea of how small a file system can be
shrunk. Right now it basically has to just start trying, and does it
one block group at a time.

Adding systemd-devel@


-- 
Chris Murphy