We tried for 4K on x86-64, too, for b quite a while as I recall. The kernel stack is a one of the main costs for a thread. I would like to decouple struct thread_info from the kernel stack (PJ Waskewicz was working on that before he left Intel) but that doesn't buy us all that much. 8K additional per thread is a huge hit. XFS has indeed always been a canary, or troublespot, I suspect because it originally came from another kernel where this was not an optimization target. On May 28, 2014 7:14:01 AM PDT, Steven Rostedt <rostedt@xxxxxxxxxxx> wrote: > >This looks like something that Linus should be involved in too. He's >been critical in the past about stack usage. > >On Wed, 28 May 2014 15:53:59 +0900 >Minchan Kim <minchan@xxxxxxxxxx> wrote: > >> While I play inhouse patches with much memory pressure on qemu-kvm, >> 3.14 kernel was randomly crashed. The reason was kernel stack >overflow. >> >> When I investigated the problem, the callstack was a little bit >deeper >> by involve with reclaim functions but not direct reclaim path. >> >> I tried to diet stack size of some functions related with >alloc/reclaim >> so did a hundred of byte but overflow was't disappeard so that I >encounter >> overflow by another deeper callstack on reclaim/allocator path. >> >> Of course, we might sweep every sites we have found for reducing >> stack usage but I'm not sure how long it saves the world(surely, >> lots of developer start to add nice features which will use stack >> agains) and if we consider another more complex feature in I/O layer >> and/or reclaim path, it might be better to increase stack size( >> meanwhile, stack usage on 64bit machine was doubled compared to 32bit >> while it have sticked to 8K. Hmm, it's not a fair to me and arm64 >> already expaned to 16K. ) >> >> So, my stupid idea is just let's expand stack size and keep an eye >> toward stack consumption on each kernel functions via stacktrace of >ftrace. >> For example, we can have a bar like that each funcion shouldn't >exceed 200K >> and emit the warning when some function consumes more in runtime. >> Of course, it could make false positive but at least, it could make a >> chance to think over it. >> >> I guess this topic was discussed several time so there might be >> strong reason not to increase kernel stack size on x86_64, for me not >> knowing so Ccing x86_64 maintainers, other MM guys and virtio >> maintainers. > >I agree with Boris that if this goes in, it should be a config option. >Or perhaps selected by those file systems that need it. I hate to have >16K stacks on a box that doesn't have that much memory, but also just >uses ext2. > >-- Steve > >> >> [ 1065.604404] kworker/-5766 0d..2 1071625990us : >stack_trace_call: Depth Size Location (51 entries) >> [ 1065.604404] ----- ---- -------- >> [ 1065.604404] kworker/-5766 0d..2 1071625991us : >stack_trace_call: 0) 7696 16 lookup_address+0x28/0x30 >> [ 1065.604404] kworker/-5766 0d..2 1071625991us : >stack_trace_call: 1) 7680 16 >_lookup_address_cpa.isra.3+0x3b/0x40 >> [ 1065.604404] kworker/-5766 0d..2 1071625991us : >stack_trace_call: 2) 7664 24 >__change_page_attr_set_clr+0xe0/0xb50 >> [ 1065.604404] kworker/-5766 0d..2 1071625991us : >stack_trace_call: 3) 7640 392 kernel_map_pages+0x6c/0x120 >> [ 1065.604404] kworker/-5766 0d..2 1071625992us : >stack_trace_call: 4) 7248 256 >get_page_from_freelist+0x489/0x920 >> [ 1065.604404] kworker/-5766 0d..2 1071625992us : >stack_trace_call: 5) 6992 352 >__alloc_pages_nodemask+0x5e1/0xb20 >> [ 1065.604404] kworker/-5766 0d..2 1071625992us : >stack_trace_call: 6) 6640 8 >alloc_pages_current+0x10f/0x1f0 >> [ 1065.604404] kworker/-5766 0d..2 1071625992us : >stack_trace_call: 7) 6632 168 new_slab+0x2c5/0x370 >> [ 1065.604404] kworker/-5766 0d..2 1071625992us : >stack_trace_call: 8) 6464 8 __slab_alloc+0x3a9/0x501 >> [ 1065.604404] kworker/-5766 0d..2 1071625993us : >stack_trace_call: 9) 6456 80 __kmalloc+0x1cb/0x200 >> [ 1065.604404] kworker/-5766 0d..2 1071625993us : >stack_trace_call: 10) 6376 376 vring_add_indirect+0x36/0x200 >> [ 1065.604404] kworker/-5766 0d..2 1071625993us : >stack_trace_call: 11) 6000 144 virtqueue_add_sgs+0x2e2/0x320 >> [ 1065.604404] kworker/-5766 0d..2 1071625993us : >stack_trace_call: 12) 5856 288 __virtblk_add_req+0xda/0x1b0 >> [ 1065.604404] kworker/-5766 0d..2 1071625993us : >stack_trace_call: 13) 5568 96 virtio_queue_rq+0xd3/0x1d0 >> [ 1065.604404] kworker/-5766 0d..2 1071625994us : >stack_trace_call: 14) 5472 128 >__blk_mq_run_hw_queue+0x1ef/0x440 >> [ 1065.604404] kworker/-5766 0d..2 1071625994us : >stack_trace_call: 15) 5344 16 blk_mq_run_hw_queue+0x35/0x40 >> [ 1065.604404] kworker/-5766 0d..2 1071625994us : >stack_trace_call: 16) 5328 96 >blk_mq_insert_requests+0xdb/0x160 >> [ 1065.604404] kworker/-5766 0d..2 1071625994us : >stack_trace_call: 17) 5232 112 >blk_mq_flush_plug_list+0x12b/0x140 >> [ 1065.604404] kworker/-5766 0d..2 1071625994us : >stack_trace_call: 18) 5120 112 >blk_flush_plug_list+0xc7/0x220 >> [ 1065.604404] kworker/-5766 0d..2 1071625995us : >stack_trace_call: 19) 5008 64 >io_schedule_timeout+0x88/0x100 >> [ 1065.604404] kworker/-5766 0d..2 1071625995us : >stack_trace_call: 20) 4944 128 mempool_alloc+0x145/0x170 >> [ 1065.604404] kworker/-5766 0d..2 1071625995us : >stack_trace_call: 21) 4816 96 bio_alloc_bioset+0x10b/0x1d0 >> [ 1065.604404] kworker/-5766 0d..2 1071625995us : >stack_trace_call: 22) 4720 48 get_swap_bio+0x30/0x90 >> [ 1065.604404] kworker/-5766 0d..2 1071625995us : >stack_trace_call: 23) 4672 160 __swap_writepage+0x150/0x230 >> [ 1065.604404] kworker/-5766 0d..2 1071625996us : >stack_trace_call: 24) 4512 32 swap_writepage+0x42/0x90 >> [ 1065.604404] kworker/-5766 0d..2 1071625996us : >stack_trace_call: 25) 4480 320 shrink_page_list+0x676/0xa80 >> [ 1065.604404] kworker/-5766 0d..2 1071625996us : >stack_trace_call: 26) 4160 208 >shrink_inactive_list+0x262/0x4e0 >> [ 1065.604404] kworker/-5766 0d..2 1071625996us : >stack_trace_call: 27) 3952 304 shrink_lruvec+0x3e1/0x6a0 >> [ 1065.604404] kworker/-5766 0d..2 1071625996us : >stack_trace_call: 28) 3648 80 shrink_zone+0x3f/0x110 >> [ 1065.604404] kworker/-5766 0d..2 1071625997us : >stack_trace_call: 29) 3568 128 >do_try_to_free_pages+0x156/0x4c0 >> [ 1065.604404] kworker/-5766 0d..2 1071625997us : >stack_trace_call: 30) 3440 208 try_to_free_pages+0xf7/0x1e0 >> [ 1065.604404] kworker/-5766 0d..2 1071625997us : >stack_trace_call: 31) 3232 352 >__alloc_pages_nodemask+0x783/0xb20 >> [ 1065.604404] kworker/-5766 0d..2 1071625997us : >stack_trace_call: 32) 2880 8 >alloc_pages_current+0x10f/0x1f0 >> [ 1065.604404] kworker/-5766 0d..2 1071625997us : >stack_trace_call: 33) 2872 200 >__page_cache_alloc+0x13f/0x160 >> [ 1065.604404] kworker/-5766 0d..2 1071625998us : >stack_trace_call: 34) 2672 80 find_or_create_page+0x4c/0xb0 >> [ 1065.604404] kworker/-5766 0d..2 1071625998us : >stack_trace_call: 35) 2592 80 >ext4_mb_load_buddy+0x1e9/0x370 >> [ 1065.604404] kworker/-5766 0d..2 1071625998us : >stack_trace_call: 36) 2512 176 >ext4_mb_regular_allocator+0x1b7/0x460 >> [ 1065.604404] kworker/-5766 0d..2 1071625998us : >stack_trace_call: 37) 2336 128 >ext4_mb_new_blocks+0x458/0x5f0 >> [ 1065.604404] kworker/-5766 0d..2 1071625998us : >stack_trace_call: 38) 2208 256 >ext4_ext_map_blocks+0x70b/0x1010 >> [ 1065.604404] kworker/-5766 0d..2 1071625999us : >stack_trace_call: 39) 1952 160 ext4_map_blocks+0x325/0x530 >> [ 1065.604404] kworker/-5766 0d..2 1071625999us : >stack_trace_call: 40) 1792 384 ext4_writepages+0x6d1/0xce0 >> [ 1065.604404] kworker/-5766 0d..2 1071625999us : >stack_trace_call: 41) 1408 16 do_writepages+0x23/0x40 >> [ 1065.604404] kworker/-5766 0d..2 1071625999us : >stack_trace_call: 42) 1392 96 >__writeback_single_inode+0x45/0x2e0 >> [ 1065.604404] kworker/-5766 0d..2 1071625999us : >stack_trace_call: 43) 1296 176 >writeback_sb_inodes+0x2ad/0x500 >> [ 1065.604404] kworker/-5766 0d..2 1071626000us : >stack_trace_call: 44) 1120 80 >__writeback_inodes_wb+0x9e/0xd0 >> [ 1065.604404] kworker/-5766 0d..2 1071626000us : >stack_trace_call: 45) 1040 160 wb_writeback+0x29b/0x350 >> [ 1065.604404] kworker/-5766 0d..2 1071626000us : >stack_trace_call: 46) 880 208 >bdi_writeback_workfn+0x11c/0x480 >> [ 1065.604404] kworker/-5766 0d..2 1071626000us : >stack_trace_call: 47) 672 144 process_one_work+0x1d2/0x570 >> [ 1065.604404] kworker/-5766 0d..2 1071626000us : >stack_trace_call: 48) 528 112 worker_thread+0x116/0x370 >> [ 1065.604404] kworker/-5766 0d..2 1071626001us : >stack_trace_call: 49) 416 240 kthread+0xf3/0x110 >> [ 1065.604404] kworker/-5766 0d..2 1071626001us : >stack_trace_call: 50) 176 176 ret_from_fork+0x7c/0xb0 >> >> Signed-off-by: Minchan Kim <minchan@xxxxxxxxxx> >> --- >> arch/x86/include/asm/page_64_types.h | 2 +- >> 1 file changed, 1 insertion(+), 1 deletion(-) >> >> diff --git a/arch/x86/include/asm/page_64_types.h >b/arch/x86/include/asm/page_64_types.h >> index 8de6d9cf3b95..678205195ae1 100644 >> --- a/arch/x86/include/asm/page_64_types.h >> +++ b/arch/x86/include/asm/page_64_types.h >> @@ -1,7 +1,7 @@ >> #ifndef _ASM_X86_PAGE_64_DEFS_H >> #define _ASM_X86_PAGE_64_DEFS_H >> >> -#define THREAD_SIZE_ORDER 1 >> +#define THREAD_SIZE_ORDER 2 >> #define THREAD_SIZE (PAGE_SIZE << THREAD_SIZE_ORDER) >> #define CURRENT_MASK (~(THREAD_SIZE - 1)) >> -- Sent from my mobile phone. Please pardon brevity and lack of formatting. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@xxxxxxxxx. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@xxxxxxxxx";> email@xxxxxxxxx </a>