> On Sep 13, 2023, at 2:47 AM, Feng Tang <feng.tang@xxxxxxxxx> wrote: > > On Tue, Sep 12, 2023 at 11:14:42PM +0800, Tang, Feng wrote: > [...] >>> >>> Well that's the problem. Since I can't run the reproducer, there's >>> nothing I can do to troubleshoot the problem myself. >> >> We dug more into the perf and other profiling data from 0Day server >> running this case, and it seems that the new simple_offset_add() >> called by shmem_mknod() brings extra cost related with slab, >> specifically the 'radix_tree_node', which cause the regression. >> >> Here is some slabinfo diff for commit a2e459555c5f and its parent: >> >> 23a31d87645c6527 a2e459555c5f9da3e619b7e47a6 >> ---------------- --------------------------- >> >> 26363 +40.2% 36956 slabinfo.radix_tree_node.active_objs >> 941.00 +40.4% 1321 slabinfo.radix_tree_node.active_slabs >> 26363 +40.3% 37001 slabinfo.radix_tree_node.num_objs >> 941.00 +40.4% 1321 slabinfo.radix_tree_node.num_slabs >> >> Also the perf profile show some difference >> >> 0.01 ±223% +0.1 0.10 ± 28% pp.self.shuffle_freelist >> 0.00 +0.1 0.11 ± 40% pp.self.xas_create >> 0.00 +0.1 0.12 ± 27% pp.self.xas_find_marked >> 0.00 +0.1 0.14 ± 18% pp.self.xas_alloc >> 0.03 ±103% +0.1 0.17 ± 29% pp.self.xas_descend >> 0.00 +0.2 0.16 ± 23% pp.self.xas_expand >> 0.10 ± 22% +0.2 0.27 ± 16% pp.self.rcu_segcblist_enqueue >> 0.92 ± 35% +0.3 1.22 ± 11% pp.self.kmem_cache_free >> 0.00 +0.4 0.36 ± 16% pp.self.xas_store >> 0.32 ± 30% +0.4 0.71 ± 12% pp.self.__call_rcu_common >> 0.18 ± 27% +0.5 0.65 ± 8% pp.self.kmem_cache_alloc_lru >> 0.36 ± 79% +0.6 0.96 ± 15% pp.self.__slab_free >> 0.00 +0.8 0.80 ± 14% pp.self.radix_tree_node_rcu_free >> 0.00 +1.0 1.01 ± 16% pp.self.radix_tree_node_ctor >> >> Some perf profile from a2e459555c5f is: >> >> - 17.09% 0.09% singleuser [kernel.kallsyms] [k] path_openat >> - 16.99% path_openat >> - 12.23% open_last_lookups >> - 11.33% lookup_open.isra.0 >> - 9.05% shmem_mknod >> - 5.11% simple_offset_add >> - 4.95% __xa_alloc_cyclic >> - 4.88% __xa_alloc >> - 4.76% xas_store >> - xas_create >> - 2.40% xas_expand.constprop.0 >> - 2.01% xas_alloc >> - kmem_cache_alloc_lru >> - 1.28% ___slab_alloc >> - 1.22% allocate_slab >> - 1.19% shuffle_freelist >> - 1.04% setup_object >> radix_tree_node_ctor >> >> Please let me know if you need more info. >> >>> >>> Is there any hope in getting this reproducer to run on Fedora? >> >> Myself haven't succeeded to reproduce it locally, will keep trying >> it tomorrow. > > It can be reproduced on a local machien with CentOS 9 (similar to > Fedora ?), Excellent, I appreciate this! > and some steps as: > > * download source code > $ wget https://sourceforge.net/projects/aimbench/files/aim-suite9/Initial%20release/s9110.tar.Z > > * untar the file and run 'make' > > * create 2 files 's9workfile' and 'test.config' > > $ cat s9workfile > # @(#) s9workfile:1.2 1/22/96 00:00:00 > # AIM Independent Resource Benchmark - Suite IX Workfile > FILESIZE: 5M > disk_src > > $ cat test.config > test > disk_src > 200s > /dev/shm/ > > * run the test with cmd "./singleuser -nl < test.config" > > The test case here is 'disk_src', so I picked one file 'disk_src.c' > and attached it for quick reference. > > The kernel config of my CentOS is different from what 0Day used, so > the perf-profile and peformance score are a little different, but > the regression trend is the same, that commit a2e459555c5f has about > 20% drop. > > Also the test platform doesn't matter, I tried on several generation > of Xeon servers which can all reproduce it. -- Chuck Lever
