This is a note to let you know that I've just added the patch titled
btrfs: account for new extents being deleted in total_bytes_pinned
to the 5.11-stable tree which can be found at:
http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary
The filename of the patch is:
btrfs-account-for-new-extents-being-deleted-in-total_bytes_pinned.patch
and it can be found in the queue-5.11 subdirectory.
If you, or anyone else, feels it should not be added to the stable tree,
please let <stable@xxxxxxxxxxxxxxx> know about it.
>From 81e75ac74ecba929d1e922bf93f9fc467232e39f Mon Sep 17 00:00:00 2001
From: Josef Bacik <josef@xxxxxxxxxxxxxx>
Date: Fri, 15 Jan 2021 16:48:56 -0500
Subject: btrfs: account for new extents being deleted in total_bytes_pinned
From: Josef Bacik <josef@xxxxxxxxxxxxxx>
commit 81e75ac74ecba929d1e922bf93f9fc467232e39f upstream.
My recent patch set "A variety of lock contention fixes", found here
https://lore.kernel.org/linux-btrfs/cover.1608319304.git.josef@xxxxxxxxxxxxxx/
(Tracked in https://github.com/btrfs/linux/issues/86)
that reduce lock contention on the extent root by running delayed refs
less often resulted in a regression in generic/371. This test
fallocate()'s the fs until it's full, deletes all the files, and then
tries to fallocate() until full again.
Before these patches we would run all of the delayed refs during
flushing, and then would commit the transaction because we had plenty of
pinned space to recover in order to allocate. However my patches made
it so we weren't running the delayed refs as aggressively, which meant
that we appeared to have less pinned space when we were deciding to
commit the transaction.
We use the space_info->total_bytes_pinned to approximate how much space
we have pinned. It's approximate because if we remove a reference to an
extent we may free it, but there may be more references to it than we
know of at that point, but we account it as pinned at the creation time,
and then it's properly accounted when the delayed ref runs.
The way we account for pinned space is if the
delayed_ref_head->total_ref_mod is < 0, because that is clearly a
freeing option. However there is another case, and that is where
->total_ref_mod == 0 && ->must_insert_reserved == 1.
When we allocate a new extent, we have ->total_ref_mod == 1 and we have
->must_insert_reserved == 1. This is used to indicate that it is a
brand new extent and will need to have its extent entry added before we
modify any references on the delayed ref head. But if we subsequently
remove that extent reference, our ->total_ref_mod will be 0, and that
space will be pinned and freed. Accounting for this case properly
allows for generic/371 to pass with my delayed refs patches applied.
It's important to note that this problem exists without the referenced
patches, it just was uncovered by them.
CC: stable@xxxxxxxxxxxxxxx # 5.10
Reviewed-by: Nikolay Borisov <nborisov@xxxxxxxx>
Signed-off-by: Josef Bacik <josef@xxxxxxxxxxxxxx>
Signed-off-by: David Sterba <dsterba@xxxxxxxx>
Signed-off-by: Greg Kroah-Hartman <gregkh@xxxxxxxxxxxxxxxxxxx>
---
fs/btrfs/delayed-ref.c | 5 +++++
fs/btrfs/extent-tree.c | 33 +++++++++++++++++++--------------
2 files changed, 24 insertions(+), 14 deletions(-)
--- a/fs/btrfs/delayed-ref.c
+++ b/fs/btrfs/delayed-ref.c
@@ -732,11 +732,16 @@ static noinline void update_existing_hea
* 2. We were negative and went to 0 or positive, so no longer can say
* that the space would be pinned, decrement our counter from the
* total_bytes_pinned counter.
+ * 3. We are now at 0 and have ->must_insert_reserved set, which means
+ * this was a new allocation and then we dropped it, and thus must
+ * add our space to the total_bytes_pinned counter.
*/
if (existing->total_ref_mod < 0 && old_ref_mod >= 0)
btrfs_mod_total_bytes_pinned(fs_info, flags, existing->num_bytes);
else if (existing->total_ref_mod >= 0 && old_ref_mod < 0)
btrfs_mod_total_bytes_pinned(fs_info, flags, -existing->num_bytes);
+ else if (existing->total_ref_mod == 0 && existing->must_insert_reserved)
+ btrfs_mod_total_bytes_pinned(fs_info, flags, existing->num_bytes);
spin_unlock(&existing->lock);
}
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -1754,23 +1754,28 @@ void btrfs_cleanup_ref_head_accounting(s
{
int nr_items = 1; /* Dropping this ref head update. */
- if (head->total_ref_mod < 0) {
+ /*
+ * We had csum deletions accounted for in our delayed refs rsv, we need
+ * to drop the csum leaves for this update from our delayed_refs_rsv.
+ */
+ if (head->total_ref_mod < 0 && head->is_data) {
+ spin_lock(&delayed_refs->lock);
+ delayed_refs->pending_csums -= head->num_bytes;
+ spin_unlock(&delayed_refs->lock);
+ nr_items += btrfs_csum_bytes_to_leaves(fs_info, head->num_bytes);
+ }
+
+ /*
+ * We were dropping refs, or had a new ref and dropped it, and thus must
+ * adjust down our total_bytes_pinned, the space may or may not have
+ * been pinned and so is accounted for properly in the pinned space by
+ * now.
+ */
+ if (head->total_ref_mod < 0 ||
+ (head->total_ref_mod == 0 && head->must_insert_reserved)) {
u64 flags = btrfs_ref_head_to_space_flags(head);
btrfs_mod_total_bytes_pinned(fs_info, flags, -head->num_bytes);
-
- /*
- * We had csum deletions accounted for in our delayed refs rsv,
- * we need to drop the csum leaves for this update from our
- * delayed_refs_rsv.
- */
- if (head->is_data) {
- spin_lock(&delayed_refs->lock);
- delayed_refs->pending_csums -= head->num_bytes;
- spin_unlock(&delayed_refs->lock);
- nr_items += btrfs_csum_bytes_to_leaves(fs_info,
- head->num_bytes);
- }
}
btrfs_delayed_refs_rsv_release(fs_info, nr_items);
Patches currently in stable-queue which might be from josef@xxxxxxxxxxxxxx are
queue-5.11/btrfs-abort-the-transaction-if-we-fail-to-inc-ref-in-btrfs_copy_root.patch
queue-5.11/btrfs-handle-space_info-total_bytes_pinned-inside-the-delayed-ref-itself.patch
queue-5.11/btrfs-splice-remaining-dirty_bg-s-onto-the-transaction-dirty-bg-list.patch
queue-5.11/btrfs-do-not-warn-if-we-can-t-find-the-reloc-root-when-looking-up-backref.patch
queue-5.11/btrfs-fix-reloc-root-leak-with-0-ref-reloc-roots-on-recovery.patch
queue-5.11/btrfs-do-not-cleanup-upper-nodes-in-btrfs_backref_cleanup_node.patch
queue-5.11/btrfs-account-for-new-extents-being-deleted-in-total_bytes_pinned.patch
queue-5.11/proc-use-kvzalloc-for-our-kernel-buffer.patch
queue-5.11/btrfs-add-asserts-for-deleting-backref-cache-nodes.patch
