On Thu, Mar 30, 2017 at 05:07:08PM +0200, Jesper Dangaard Brouer wrote: > On Thu, 30 Mar 2017 14:04:36 +0100 > Mel Gorman <mgorman@xxxxxxxxxxxxxxxxxxx> wrote: > > > On Wed, Mar 29, 2017 at 09:44:41PM +0200, Jesper Dangaard Brouer wrote: > > > > Regardless or using in_irq() (or in combi with in_nmi()) I get the > > > > following warning below: > > > > > > > > [ 0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-4.11.0-rc3-net-next-page-alloc-softirq+ root=UUID=2e8451ff-6797-49b5-8d3a-eed5a42d7dc9 ro rhgb quiet LANG=en_DK.UTF > > > > -8 > > > > [ 0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes) > > > > [ 0.000000] ------------[ cut here ]------------ > > > > [ 0.000000] WARNING: CPU: 0 PID: 0 at kernel/softirq.c:161 __local_bh_enable_ip+0x70/0x90 > > > > [ 0.000000] Modules linked in: > > > > [ 0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted 4.11.0-rc3-net-next-page-alloc-softirq+ #235 > > > > [ 0.000000] Hardware name: MSI MS-7984/Z170A GAMING PRO (MS-7984), BIOS 1.60 12/16/2015 > > > > [ 0.000000] Call Trace: > > > > [ 0.000000] dump_stack+0x4f/0x73 > > > > [ 0.000000] __warn+0xcb/0xf0 > > > > [ 0.000000] warn_slowpath_null+0x1d/0x20 > > > > [ 0.000000] __local_bh_enable_ip+0x70/0x90 > > > > [ 0.000000] free_hot_cold_page+0x1a4/0x2f0 > > > > [ 0.000000] __free_pages+0x1f/0x30 > > > > [ 0.000000] __free_pages_bootmem+0xab/0xb8 > > > > [ 0.000000] __free_memory_core+0x79/0x91 > > > > [ 0.000000] free_all_bootmem+0xaa/0x122 > > > > [ 0.000000] mem_init+0x71/0xa4 > > > > [ 0.000000] start_kernel+0x1e5/0x3f1 > > > > [ 0.000000] x86_64_start_reservations+0x2a/0x2c > > > > [ 0.000000] x86_64_start_kernel+0x178/0x18b > > > > [ 0.000000] start_cpu+0x14/0x14 > > > > [ 0.000000] ? start_cpu+0x14/0x14 > > > > [ 0.000000] ---[ end trace a57944bec8fc985c ]--- > > > > [ 0.000000] Memory: 32739472K/33439416K available (7624K kernel code, 1528K rwdata, 3168K rodata, 1860K init, 2260K bss, 699944K reserved, 0K cma-reserved) > > > > > > > > And kernel/softirq.c:161 contains: > > > > > > > > WARN_ON_ONCE(in_irq() || irqs_disabled()); > > > > > > > > Thus, I don't think the change in my RFC-patch[1] is safe. > > > > Of changing[2] to support softirq allocations by replacing > > > > preempt_disable() with local_bh_disable(). > > > > > > > > [1] http://lkml.kernel.org/r/20170327143947.4c237e54@xxxxxxxxxx > > > > > > > > [2] commit 374ad05ab64d ("mm, page_alloc: only use per-cpu allocator for irq-safe requests") > > > > https://git.kernel.org/torvalds/c/374ad05ab64d > > > > > > A patch that avoids the above warning is inlined below, but I'm not > > > sure if this is best direction. Or we should rather consider reverting > > > part of commit 374ad05ab64d to avoid the softirq performance regression? > > > > > > > At the moment, I'm not seeing a better alternative. If this works, I > > think it would still be far superior in terms of performance than a > > revert. > > Started performance benchmarking: > 163 cycles = current state > 183 cycles = with BH disable + in_irq > 218 cycles = with BH disable + in_irq + irqs_disabled > > Thus, the performance numbers unfortunately looks bad, once we add the > test for irqs_disabled(). The slowdown by replacing preempt_disable > with BH-disable is still a win (we saved 29 cycles before, and loose > 20, I was expecting regression to be only 10 cycles). > This surprises me because I'm not seeing the same severity of problems with irqs_disabled. Your path is slower than what's currently upstream but it's still far better than a revert. The softirq column in the middle is your patch versus a full revert which is the last columnm 4.11.0-rc5 4.11.0-rc5 4.11.0-rc5 vanilla softirq-v2r1 revert-v2r1 Amean alloc-odr0-1 217.00 ( 0.00%) 223.00 ( -2.76%) 280.54 (-29.28%) Amean alloc-odr0-2 162.23 ( 0.00%) 174.46 ( -7.54%) 210.54 (-29.78%) Amean alloc-odr0-4 144.15 ( 0.00%) 150.38 ( -4.32%) 182.38 (-26.52%) Amean alloc-odr0-8 126.00 ( 0.00%) 132.15 ( -4.88%) 282.08 (-123.87%) Amean alloc-odr0-16 117.00 ( 0.00%) 122.00 ( -4.27%) 253.00 (-116.24%) Amean alloc-odr0-32 113.00 ( 0.00%) 118.00 ( -4.42%) 145.00 (-28.32%) Amean alloc-odr0-64 110.77 ( 0.00%) 114.31 ( -3.19%) 143.00 (-29.10%) Amean alloc-odr0-128 109.00 ( 0.00%) 107.69 ( 1.20%) 179.54 (-64.71%) Amean alloc-odr0-256 121.00 ( 0.00%) 125.00 ( -3.31%) 232.23 (-91.93%) Amean alloc-odr0-512 123.46 ( 0.00%) 129.46 ( -4.86%) 148.08 (-19.94%) Amean alloc-odr0-1024 123.23 ( 0.00%) 128.92 ( -4.62%) 142.46 (-15.61%) Amean alloc-odr0-2048 125.92 ( 0.00%) 129.62 ( -2.93%) 147.46 (-17.10%) Amean alloc-odr0-4096 133.85 ( 0.00%) 139.77 ( -4.43%) 155.69 (-16.32%) Amean alloc-odr0-8192 138.08 ( 0.00%) 142.92 ( -3.51%) 159.00 (-15.15%) Amean alloc-odr0-16384 133.08 ( 0.00%) 140.08 ( -5.26%) 157.38 (-18.27%) Amean alloc-odr1-1 390.27 ( 0.00%) 401.53 ( -2.89%) 389.73 ( 0.14%) Amean alloc-odr1-2 306.33 ( 0.00%) 311.07 ( -1.55%) 304.07 ( 0.74%) Amean alloc-odr1-4 250.87 ( 0.00%) 258.00 ( -2.84%) 256.53 ( -2.26%) Amean alloc-odr1-8 221.00 ( 0.00%) 231.07 ( -4.56%) 221.20 ( -0.09%) Amean alloc-odr1-16 212.07 ( 0.00%) 223.07 ( -5.19%) 208.00 ( 1.92%) Amean alloc-odr1-32 210.07 ( 0.00%) 215.20 ( -2.44%) 208.20 ( 0.89%) Amean alloc-odr1-64 197.00 ( 0.00%) 203.00 ( -3.05%) 203.00 ( -3.05%) Amean alloc-odr1-128 204.07 ( 0.00%) 189.27 ( 7.25%) 200.00 ( 1.99%) Amean alloc-odr1-256 193.33 ( 0.00%) 190.53 ( 1.45%) 193.80 ( -0.24%) Amean alloc-odr1-512 180.60 ( 0.00%) 190.33 ( -5.39%) 183.13 ( -1.40%) Amean alloc-odr1-1024 176.93 ( 0.00%) 182.40 ( -3.09%) 176.33 ( 0.34%) Amean alloc-odr1-2048 184.60 ( 0.00%) 191.33 ( -3.65%) 180.60 ( 2.17%) Amean alloc-odr1-4096 184.80 ( 0.00%) 182.60 ( 1.19%) 182.27 ( 1.37%) Amean alloc-odr1-8192 183.60 ( 0.00%) 180.93 ( 1.45%) 181.07 ( 1.38%) I revisisted having an irq-safe list but it's excessively complex and there are significant problems where it's not clear it can be handled safely so it's not a short-term option. -- Mel Gorman SUSE Labs -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@xxxxxxxxx. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@xxxxxxxxx";> email@xxxxxxxxx </a>
