On Mon 18-03-24 18:09:18, Ojaswin Mujoo wrote:
> On Thu, Mar 14, 2024 at 10:09:01PM +0800, Baokun Li wrote:
> > We can trigger a slab-out-of-bounds with the following commands:
> >
> > mkfs.ext4 -F /dev/$disk 10G
> > mount /dev/$disk /tmp/test
> > echo 2147483647 > /sys/fs/ext4/$disk/mb_group_prealloc
> > echo test > /tmp/test/file && sync
> >
> > ==================================================================
> > BUG: KASAN: slab-out-of-bounds in ext4_mb_find_good_group_avg_frag_lists+0x8a/0x200 [ext4]
> > Read of size 8 at addr ffff888121b9d0f0 by task kworker/u2:0/11
> > CPU: 0 PID: 11 Comm: kworker/u2:0 Tainted: GL 6.7.0-next-20240118 #521
> > Call Trace:
> > dump_stack_lvl+0x2c/0x50
> > kasan_report+0xb6/0xf0
> > ext4_mb_find_good_group_avg_frag_lists+0x8a/0x200 [ext4]
> > ext4_mb_regular_allocator+0x19e9/0x2370 [ext4]
> > ext4_mb_new_blocks+0x88a/0x1370 [ext4]
> > ext4_ext_map_blocks+0x14f7/0x2390 [ext4]
> > ext4_map_blocks+0x569/0xea0 [ext4]
> > ext4_do_writepages+0x10f6/0x1bc0 [ext4]
> > [...]
> > ==================================================================
> >
> > The flow of issue triggering is as follows:
> >
> > // Set s_mb_group_prealloc to 2147483647 via sysfs
> > ext4_mb_new_blocks
> > ext4_mb_normalize_request
> > ext4_mb_normalize_group_request
> > ac->ac_g_ex.fe_len = EXT4_SB(sb)->s_mb_group_prealloc
> > ext4_mb_regular_allocator
> > ext4_mb_choose_next_group
> > ext4_mb_choose_next_group_best_avail
> > mb_avg_fragment_size_order
> > order = fls(len) - 2 = 29
> > ext4_mb_find_good_group_avg_frag_lists
> > frag_list = &sbi->s_mb_avg_fragment_size[order]
> > if (list_empty(frag_list)) // Trigger SOOB!
> >
> > At 4k block size, the length of the s_mb_avg_fragment_size list is 14,
> > but an oversized s_mb_group_prealloc is set, causing slab-out-of-bounds
> > to be triggered by an attempt to access an element at index 29.
> >
> > Add a new attr_id attr_clusters_in_group with values in the range
> > [0, sbi->s_clusters_per_group] and declare mb_group_prealloc as
> > that type to fix the issue. In addition avoid returning an order
> > from mb_avg_fragment_size_order() greater than MB_NUM_ORDERS(sb)
> > and reduce some useless loops.
> >
> > Fixes: 7e170922f06b ("ext4: Add allocation criteria 1.5 (CR1_5)")
> > CC: stable@xxxxxxxxxxxxxxx
> > Signed-off-by: Baokun Li <libaokun1@xxxxxxxxxx>
> > Reviewed-by: Jan Kara <jack@xxxxxxx>
> > ---
> > fs/ext4/mballoc.c | 4 ++++
> > fs/ext4/sysfs.c | 13 ++++++++++++-
> > 2 files changed, 16 insertions(+), 1 deletion(-)
> >
> > diff --git a/fs/ext4/mballoc.c b/fs/ext4/mballoc.c
> > index 12b3f196010b..48afe5aa228c 100644
> > --- a/fs/ext4/mballoc.c
> > +++ b/fs/ext4/mballoc.c
> > @@ -831,6 +831,8 @@ static int mb_avg_fragment_size_order(struct super_block *sb, ext4_grpblk_t len)
> > return 0;
> > if (order == MB_NUM_ORDERS(sb))
> > order--;
> > + if (WARN_ON_ONCE(order > MB_NUM_ORDERS(sb)))
> > + order = MB_NUM_ORDERS(sb) - 1;
>
> Hey Baokun,
>
> Thanks for fixing this. This patch looks good to me, feel free to add:
>
> Reviewed-by: Ojaswin Mujoo <ojaswin@xxxxxxxxxxxxx>
>
> my comments after this are less about the patch and more about some
> thoughts on the working of average fragment lists.
>
> So going through the v2 and this patch got me thinking about what really
> is going to happen when a user tries to allocate 32768 blocks which is also
> the maximum value we could have in say ac->ac_g_ex.fe_len.
>
> When this happens, ext4_mb_regular_allocator() will directly set the
> criteria as CR_GOAL_LEN_FAST. Now, we'll follow:
>
> ext4_mb_choose_next_group_goal_fast()
> for (i=mb_avg_fragment_size_order(); i < MB_NUM_ORDERS; i++) { .. }
>
> Here, mb_avg_fragment_siz_order() will do something like:
>
> order = fls(32768) - 2 = 14
> ...
> if (order == MB_NUM_ORDERS(sb))
> order--;
>
> return order;
>
> And we'll look in the fragment list[13] and since none of the groups
> there would have 32768 blocks free (since we dont track it here) we'll
> unnecessarily traverse the full list before falling to CR_BEST_AVAIL_LEN
> (this will become a noop due to the way order and min_order
> are calculated) and eventually to CR_GOAL_LEN_SLOW where we might get
> something or end up splitting.
Yeah, agreed this looks a bit suboptimal. I'm just not 100% sure whether
we'll ever generate a request to allocate 32768 blocks - that would need
verification with tracing - because I have some vague recollection I once
arrived at conclusion this is not possible.
> I think something more optimal would be to:
>
> 1. Add another entry to average fragment lists for completely empty
> groups. (As a sidenote i think we should use something like MB_NUM_FRAG_ORDER
> instead of MB_NUM_ORDERS in calculating limits related to average
> fragment lists since the NUM_ORDERS seems to be the buddy max order ie
> 8192 blocks only valid for CR_POWER2 and shouldn't really limit the
> fragment size lists)
I guess the thinking was that you can never get larger than
1<<(MB_NUM_ORDERS-1) chunk from mballoc so there's no point to keep
fragment lists of such chunks?
Honza
--
Jan Kara <jack@xxxxxxxx>
SUSE Labs, CR
