On 3/15/22 6:23 PM, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@xxxxxxxxxx>
>
> Recently, the upstream maintainers have been taking a lot of heat on
> account of writer threads encountering high latency when asking for log
> grant space when the log is small. The reported use case is a heavily
> threaded indexing product logging trace information to a filesystem
> ranging in size between 20 and 250GB. The meetings that result from the
> complaints about latency and stall warnings in dmesg both from this use
> case and also a large well known cloud product are now consuming 25% of
> the maintainer's weekly time and have been for months.
And we don't want that!
> For small filesystems, the log is small by default because we have
> defaulted to a ratio of 1:2048 (or even less). For grown filesystems,
> this is even worse, because big filesystems generate big metadata.
> However, the log size is still insufficient even if it is formatted at
> the larger size.
I have no complaints about raising the log size like this; I think it's prudent,
even if it doesn't solve world hunger and bring global peace.
I do want to give a little more thought to how calculate_log_size() looks now;
it's inherited spaghetti but my eye twitches a little bit when we follow this:
* For small filesystems, we want to use the
* XFS_MIN_LOG_BYTES for filesystems smaller than 16G if
* at all possible,
with "haha no actually we should calculate something realistic" ;)
I don't want to hold this up for long but I want to put a little thought into
whether the resulting code can be a bit more understandable, there's so much
pingponging around about clamping the size up, down, bigger, smaller, this
heuristic, that heuristic I'm having trouble keeping it straight in my
old brain.
Thanks,
-Eric
