On Mon, Oct 21, 2024 at 8:54 PM dexen deVries wrote:
>
> Hello list,
>
> lscp's "-g, --show-increment" option seems to give misleading number
> of appended blocks, significantly under-counting disk usage. i
> expected the sum total of increments, plus the full size of the first
> checkpoint, multiplied by block size (4096 in my case) to be equal to
> total used disk space (or slightly below it, due to partly used
> segments). and while the size of each checkpoint is indeed roughly
> equal to disk use by that checkpoint, the sum of increments
> significantly undercounts disk space usage.
>
> am i using it wrong, or is it bugged?
> using latest kernel and long-running filesystems
>
> example from one of my computers: 170GB calculated vs 356GB actual use
> Nr checkpoints: 49477
> Oldest checkpoint used space [MB]: 141903
> Computed sum increments used space [MB]: 28777
> Computed sum total used space [MB]: 170680
> Actual used space reported by df -m [MB]: 354072
>
>
> example from another of my computers: 34GB calculated vs 119GB actual use
> Nr checkpoints: 2368
> Oldest checkpoint used space [MB]: 27520
> Computed sum increments used space [MB]: 6813
> Computed sum total used space [MB]: 34334
> Actual used space reported by df -m [MB]: 119040
>
> repro script:
> #!/bin/sh
>
> fs=`df . | awk 'NR==2 {print $1}'`
> lscp $fs | awk 'END {print "Nr checkpoints: " NR-1}'
> lscp --all $fs | awk 'NR==2 {print $6}' | awk '{printf("Oldest
> checkpoint used space [MB]: %15d\n", ($1*4096)/1024/1024)}'
> lscp --all --show-increment $fs | head -n-1 | awk 'NR>2 {sum=sum+$6}
> END {print sum}' | awk '{printf("Computed sum increments used space
> [MB]: %9d\n", ($1*4096)/1024/1024)}'
> {
> lscp --all $fs | awk 'NR==2 {print $6}'
> lscp --all --show-increment $fs | head -n-1 | awk 'NR>2 {print $6}'
> } | awk '{sum=sum+$1} END { printf("Computed sum total used space
> [MB]: %14d\n", (sum*4096)/1024/1024)}'
> df -m $fs | awk 'NR==2 {printf("Actual used space reported by df -m
> [MB]: %8d\n", $3)}'
>
>
> cheers,
> --
> dexen
Dear dexen,
I've been busy this week, sorry for the late reply.
The number displayed by the "-g, --show-increment" option of lscp is
the incremental block count measured when writing logs recorded in the
checkpoint metadata, but I'm going to check the implementation to see
if it's what we intended.
Please give me a little time.
However, please do not use this number in general.
Checkpoints are deleted by GC, and the differential block count also
becomes inaccurate due to GC.
Therefore, this value of the block count is meaningless in practice.
(And that is why it's hidden and not the default.)
Use only the absolute block count displayed by default, and if
necessary, calculate the difference with the absolute value of the
previous remaining checkpoint.
If it's really necessary, it might be useful to add a block count
difference display function to the lscp command that performs such
searches and calculations internally. It's clear what needs to be
implemented, but I'm currently busy dealing with the constant reports
of issues in the kernel, so I can't say anything promising.
Regards,
Ryusuke Konishi
