On Wed, Sep 1, 2021 at 11:09 AM Shijie Huang <shijie@xxxxxxxxxxxxxxxxxxxxxxxxxx> wrote: > > Hi Everyone, > > In the NUMA, we only have one page cache for each file. For the > program/shared libraries, the > > remote-access delays longer then the local-access. > > So, is it possible to implement the per-node page cache for > programs/libraries? as far as i know, this is an very interesting topic, we do have some "solutions" on this. MIPS kernel supports kernel TEXT replication: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/mips/sgi-ip27/Kconfig https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/mips/sgi-ip27/ip27-klnuma.c config REPLICATE_KTEXT bool "Kernel text replication support" depends on SGI_IP27 select MAPPED_KERNEL help Say Y here to enable replicating the kernel text across multiple nodes in a NUMA cluster. This trades memory for speed. for x86, RedHawk Linux（https://www.concurrent-rt.com/solutions/linux/）supports kernel text replication. here are some benchmark: https://www.concurrent-rt.com/wp-content/uploads/2016/11/kernel-page-replication.pdf For userspace, dplace from SGI can help replicate text: https://www.spec.org/cpu2006/flags/SGI-platform.html -r bl: specifies that text should be replicated on the NUMA node or nodes where the process is running. 'b' indicates that binary (a.out) text should be replicated; 'l' indicates that library text should be replicated. but all of the above except mips ktext replication are out of tree. Please count me in if you have any solution and any pending patch. I am interested in this topic. > > > We can do it like this: > > 1.) Add a new system call to control specific files to > NUMA-aware, such as: > > set_numa_aware("/usr/lib/libc.so", enable); > > After the system call, the page cache of libc.so has the > flags "NUMA_ENABLED" > > > 2.) When A new process tries to setup the MMU page table for > libc.so, it will check > > if NUMA_ENABLED is set. If it set, the kernel will give a > page which is bind to the process's NUMA node. > > By this way, we can eliminate the remote-access for > programs/shared library. > > > Is this proposal ok? Or do you have a better idea? > > > Thanks > > Huang Shijie Thanks barry
