Hello All, v2 -> v3: v3 changes the code design to fix ENOSPC error. This patch uses the percpu discard pa seq counter (which was as discussed with Jan & Aneesh). Patch-2 commit msg describes both the problem and the new algorithm in great detail. Rest of the patches are mostly either refactoring, code cleanups or debug logs improvements. Posting this for early review comments. For now I have done some basic testing on this patch to test for ENOSPC reported errors, using a smaller filesystem (~240MB, 64K bs) a. Tested multi-thread file writes which only allocates group PAs. b. Tested multi-thread file writes which only allocates inode PAs. c. Tested multi-thread file writes doing combination of both of the above. [May Not be Ready for Merge yet, until below are properly discussed] ================================================================== 1. There is a query asked in Patch-2 commit msg itself, regarding rcu_barrier() usage. 2. AFAICT, even if we reduce the PA size based on available FS size to avoid this ENOSPC error, this issue could still potentially happen since the race has mostly to do with 1st thread freeing up all the PAs while 2nd thread returning 0 as freed (since PA list was empty). Hence 2nd thread fails with ENOSPC even though there were free blocks freed by 1st thread. 3. I would like to know if there is any stress-ng test case or any other use case which measures multi-thread performance of write while FS is close to ENOSPC? If yes - instead of regressing this later, I would like to know such test case so that it can be tested at my end while we are still at it. 4. I do see that with 64K blocksize the performance of multi-thread < 1 MB file size writes becomes very slow. But without this patch, it anyways fails with ENOSPC error. On doing below the performance does improve close to 5x. echo 32 > /sys/fs/ext4/loop3/mb_group_prealloc I was thinking instead of 512 blocks as default value for 'MB_DEFAULT_GROUP_PREALLOC', we could make it 64K as the default size and decide the no. of blocks based on the blocksize. But I haven't got to it yet. But thought to capture it in this email though. We can get to it after these patches. 5. Started fstests testing. Will let you know the results soon. [RFCv2]: https://patchwork.ozlabs.org/project/linux-ext4/patch/533ac1f5b19c520b08f8c99aec5baf8729185714.1586954511.git.riteshh@xxxxxxxxxxxxx/ - Problems with v2 are captured in Patch-2 commit msg itself. Ritesh Harjani (20): ext4: mballoc: Refactor ext4_mb_discard_preallocations() ext4: Introduce percpu seq counter for freeing blocks(PA) to avoid ENOSPC err ext4: mballoc: Do print bb_free info even when it is 0 ext4: mballoc: Refactor ext4_mb_show_ac() ext4: mballoc: Add more mb_debug() msgs ext4: mballoc: Correct the mb_debug() format specifier for pa_len var ext4: mballoc: Fix few other format specifier in mb_debug() ext4: mballoc: Simplify error handling in ext4_init_mballoc() ext4: mballoc: Make ext4_mb_use_preallocated() return type as bool ext4: mballoc: Remove EXT4_MB_HINT_GOAL_ONLY and it's related code ext4: mballoc: Refactor code inside DOUBLE_CHECK into separate function ext4: mballoc: Fix possible NULL ptr dereference from mb_cmp_bitmaps() ext4: mballoc: Don't BUG if kmalloc or read blk bitmap fail for DOUBLE_CHECK ext4: balloc: Use task_pid_nr() helper ext4: Use BIT() macro for BH_** state bits ext4: Improve ext_debug() msg in case of block allocation failure ext4: Replace EXT_DEBUG with __maybe_unused in ext4_ext_handle_unwritten_extents() ext4: mballoc: Make mb_debug() implementation to use pr_debug() ext4: Make ext_debug() implementation to use pr_debug() ext4: Add process name and pid in ext4_msg() fs/ext4/Kconfig | 3 +- fs/ext4/balloc.c | 5 +- fs/ext4/ext4.h | 38 ++-- fs/ext4/extents.c | 150 ++++++++-------- fs/ext4/inode.c | 15 +- fs/ext4/mballoc.c | 347 +++++++++++++++++++++++------------- fs/ext4/mballoc.h | 16 +- fs/ext4/super.c | 3 +- include/trace/events/ext4.h | 1 - 9 files changed, 335 insertions(+), 243 deletions(-) -- 2.21.0
