On Fri, Apr 15, 2022 at 12:25 AM Linus Torvalds <torvalds@xxxxxxxxxxxxxxxxxxxx> wrote: > > > And it could be that if I have 150k of those smallish allocations, a > server with lots of active users might have millions. Not having > looked at where they come from, maybe that isn't the case, but it > *might* be. > > Maybe adding something like a > > static int warn_every_1k = 0; > WARN_ON(size < 32 && (1023 & ++warn_every_1k)); > > to kmalloc() would give us a statistical view of "lots of these small > allocations" thing, and we could add GFP_NODMA to them. There probably > aren't that many places that have those small allocations, and it's > certainly safer to annotate "this is not for DMA" than have the > requirement that all DMA allocations must be marked. I think finding out the allocations is one of the most common examples for ftrace. I followed the instructions from https://www.kernel.org/doc/Documentation/trace/events.txt to show me a histogram of all allocations under 256 bytes, which (one kernel compile later) gives me something like $echo 'hist:key=call_site.sym-offset:val=bytes_req:sort=bytes_req.descending if bytes_req<256' > \ /sys/kernel/debug/tracing/events/kmem/kmalloc/trigger $ make -skj30 ... $ head /sys/kernel/debug/tracing/events/kmem/kmalloc/hist { call_site: [ffffffffc04e457f] btrfs_delete_delayed_dir_index+0xbf/0x1e0 [btrfs] } hitcount: 146914 bytes_req: 16454368 { call_site: [ffffffffbbe601a3] generic_file_buffered_read+0x463/0x4a0 } hitcount: 98187 bytes_req: 14906232 { call_site: [ffffffffc0497b81] btrfs_buffered_write+0x131/0x7e0 [btrfs] } hitcount: 156513 bytes_req: 10038544 { call_site: [ffffffffc05125c9] btrfs_alloc_block_rsv+0x29/0x60 [btrfs] } hitcount: 155044 bytes_req: 8682464 { call_site: [ffffffffbbfe7272] kernfs_fop_open+0xc2/0x290 } hitcount: 38764 bytes_req: 5892128 { call_site: [ffffffffbbfb6ea2] load_elf_binary+0x242/0xed0 } hitcount: 58276 bytes_req: 3729664 { call_site: [ffffffffc04b52d0] __btrfs_map_block+0x1f0/0xb60 [btrfs] } hitcount: 29289 bytes_req: 3521656 { call_site: [ffffffffbbf7ac7e] inotify_handle_inode_event+0x7e/0x210 } hitcount: 61688 bytes_req: 2986992 { call_site: [ffffffffbbf2fa35] alloc_pipe_info+0x65/0x230 } hitcount: 13139 bytes_req: 2312464 { call_site: [ffffffffbc0cd3ec] security_task_alloc+0x9c/0x100 } hitcount: 60475 bytes_req: 2177100 { call_site: [ffffffffbc0cd5f6] security_prepare_creds+0x76/0xa0 } hitcount: 266124 bytes_req: 2128992 { call_site: [ffffffffbbfe710e] kernfs_get_open_node+0x7e/0x120 } hitcount: 38764 bytes_req: 1860672 { call_site: [ffffffffc04e1fbd] btrfs_alloc_delayed_item+0x1d/0x50 [btrfs] } hitcount: 11859 bytes_req: 1833383 { call_site: [ffffffffc046595d] split_item+0x8d/0x2e0 [btrfs] } hitcount: 14049 bytes_req: 1716288 { call_site: [ffffffffbbfb6dbc] load_elf_binary+0x15c/0xed0 } hitcount: 58276 bytes_req: 1631728 { call_site: [ffffffffbbf40e79] __d_alloc+0x179/0x1f0 } hitcount: 24814 bytes_req: 1280649 { call_site: [ffffffffbbf5203f] single_open+0x2f/0xa0 } hitcount: 34541 bytes_req: 1105312 { call_site: [ffffffffc047ad0a] btrfs_wq_submit_bio+0x4a/0xe0 [btrfs] } hitcount: 7746 bytes_req: 1053456 { call_site: [ffffffffbc519e95] xhci_urb_enqueue+0xf5/0x3c0 } hitcount: 5511 bytes_req: 484968 { call_site: [ffffffffc0482935] btrfs_opendir+0x25/0x70 [btrfs] } hitcount: 60245 bytes_req: 481960 { call_site: [ffffffffc04c44ff] overwrite_item+0x1cf/0x5c0 [btrfs] } hitcount: 7378 bytes_req: 364305 { call_site: [ffffffffc04c4514] overwrite_item+0x1e4/0x5c0 [btrfs] } hitcount: 7378 bytes_req: 364305 { call_site: [ffffffffc04e207f] btrfs_wq_run_delayed_node+0x2f/0x80 [btrfs] } hitcount: 3427 bytes_req: 356408 { call_site: [ffffffffbbe7e96d] shmem_symlink+0xbd/0x250 } hitcount: 5169 bytes_req: 242943 { call_site: [ffffffffc03e0526] hid_input_field+0x56/0x290 [hid] } hitcount: 11004 bytes_req: 175760 I think these are all safe for the GFP_NODMA approach you suggest, maybe not the xhci_urb_enqueue one. Arnd
