On 4/27/06, Hayim Shaul <hayim@xxxxxxxxxxxxxx> wrote: > > I delved a little in my "Understanding the Linux Kernel". It's the first > edition, but I guess it's a starting point. Here's what I found ... > > > >> > >>> mem_map is an array of page structure, each element of this array is > >>> of "page" structure type. Actually mem_map is used for reverse > >>> mappeing of physical memory of system. This array represents the > >>> physical memory interms of page sturcture. Lets say for simplicity we > >>> have total physical memory of 256K and one one physical page of 4K, > >>> then in taht case mem_map will have 256/4 = 64 enteries and max_mapnr > >>> will be set to 64. > >>> > > Linux keeps track of each frame in the system. Each frame has a structure > keeping some info about it. The table of all these structure is the > mem_map. (as you said.) > > >>> So as per the kernel sources (file: include/asm-i386/page.h) > >>> #define pfn_valid(pfn) ((pfn) < max_mapnr) > >>> #define virt_addr_valid(kaddr) pfn_valid(__pa(kaddr) >> PAGE_SHIFT) > >>> > >>> virt_addr_valid() macro only tells whether the given virtual address > >>> can be mapped to any physical page or not. __pa() macro gives the > >>> physical address for a given virtual address just by subracting the > >>> PAGE_OFFSET from given virtual address. Now then we right shift that > >>> physical address by page size which gives us the page number in which > >>> our physical adress falls and then we look if this page number is less > >>> than the total number of physical pages in system, if yes then it > >>> means we can mapped this virtual address to some physical address else > >>> our given virtual address can not be mapped to any physical address > >>> in system. > >> > >> I think I get it. You don't have more pages than RAM frames (can't the > >> kernel swap its own pages?) so all this check does is make sure the > >> address falls in a page you are allowed to use. > > My statement here is not correct. > You can have (much) more pages than frames. > I am not sure whether Linux swaps kernel pages, but some pages can be > swapped out theoretically speaking. I dont think kernel memory can ever be swapped out, but definitely user process memory is swappable. If I am wrong correct me. > > > As kernel occupies contiguous physical memory, virt_addr_valid() only > > tells if this virtual address can be mapped to some physical location > > in RAM or is it going outside that (means system dont have that much > > RAM so that it can map this large virtual address). Lets say we have > > pass vitual address 0xC0080000 (PAGE_OFFSET + 512 K) and we only have > > physical RAM of 256K (just for keeping the example simple). in this > > case total number of physical pages will be 256 / 4 = 64 (each page > > being 4K) and the page frame number we will get for our page will be > > ((virtual address - PAGE_OFFSET) >> 12) = 128. This means our virtual > > address will be there in 128th page frame, but 128th page frame does > > not exist in our RAM, so its not a valid or rather not at all possible > > mappable virtual address. Hope with this example it clears that > > virt_addr_valid() function only tells you if virtual address can be > > mapped to some physical page or not and nothing more than that. > > > > AFAIK, kernel memory is never swapped out. > > What bothered be was the macro __pa() that translates virtual to physical > addresses by simple substruction. This would imply that virtual > addresses are contiguous in RAM, external fragmentation, and other ouchies. kernel virtual address space is is not fragmented in physical memory, its contigous. This does not mean that kernel do not need the page table for it self. Kernel do have the page table for its virtual to physical memory mapping as that is needed by CPU while addresing in virtual addressing mode. Kernel page table maps the contigous kernel virtual address space to contigous physical address space, the only difference is that offset 0xC0080000 (PAGE_OFFSET) in virtuall address space is mapped to physical address 0x00000000. That is why in whenever reffering to kernel virtual address we just need to subtract PAGE_OFFSET from it to get the corresponding physical address. NOTE: this is not at all true for user space virtual address. mapping of user space virtual address is always done through traversal of page table for that process, as the virtual address of user space process can be fragmented in terms of physical pages. Hope all this does not confuse you further ;-) ..... well its bit confusing at initial stage. I am also not much confident in all this, so experts on list, please correct me if I am wrong anywhere :-) > > I read some and eventually saw that: > The kernel has 1G of virtual addresses (usually). The lower part of this > 1G is used to map the physical memory. The upper part of it is used for > virtual addresses. > So we have this mem-map: > > [ map of ram | 8MB | more kernel memory ] > ^ ^ ^ ^ > PAGE_OFFSET high_memory VMALLOC_START 4G > > Where PAGE_OFFSET is 0xC0000000. > > high_memory is a variable that tells where the mapping of physical RAM > ends. > > VMALLOC_START points to where starts the area that is allocated virtually. > > So, addresses between PAGE_OFFSET and high_memory can be translated with > __pa(), others cannot. > Not sure of this ...... But I know that in physical memory first 8K of memory is left blank as a hole, this means the first virtual address of kernel will be mapped to 8K and not 0 physical address. The reason for this is given in source code itself: http://lxr.linux.no/source/include/asm-i386/pgtable.h#L72 As mentioned in source code also, this is done to catch any mistakes of null pointer de-reffrening in kernel code. > >> > >> > >> So if we go back to Talib's original question, the only thing he can do is > >> traverse the page table tree to see if the address is indeed valid. > >> I think this is what virt_to_page(addr) does. > > > > I dont think he need to traverse the page tables as he has to verify > > the kernel virtual address not the user virtual address. He can > > definitely use virt_addr_valid() to make sure that the passed virtual > > address is atleast mappable to some physical address in RAM and then > > over that use his signature and pid matching mechanisum to make sure > > that its the same memory location whose pointer he passed back to user > > earlier. > > I am not sure how Talib allocated his structures. If it's with kmalloc the > __pa() and virt_to_page() can work for him. And I guess these addresses > are always valid as they map the RAM (which is always there hopefully). If these are modified by user space application and the passed to kernel, they might not be mappable to some physical page frame, so need to cross check. > > If its with vmalloc then he probably needs to traverse the page-table > structure. Yes, you are right, if vmalloc is being used then page table for kernel virtual memory mapping should be used. > > >>> Having understood all this, and applying this information to the > >>> actual problem of verifying the virtual address, we can use this macro > >>> to just make sure that the kernel virtual address passed by user space > >>> back to kernel falls in kernel virtual address space but, we can not > >>> be sure if the given address is the same as we passed it to user space > >>> earlier, for that as mentioned by some one we need to user some > >>> ginature in the structure being pointed by this virtual address. > >> > >> You have to check first that the virtual address is mapped to real a > >> frame, or you'll get a kernel panic. no? > >> > > What I still don't understand is how is this mapping happens on machines > where the kernel space is of size 1G and they have more than 1G of RAM. > > Probably only a small portion of the RAM is mapped. Please elaborate more on this, I could not get that want to say by this. regards, Gaurav > > Does what I wrote make sense, or am I so tied up with my own > un-understanding that I can't even notice it? > > > Hayim. > -- -- -Gaurav Email: gauravd.chd@xxxxxxxxx --------------------------------- Read my blog at: http://lkdp.blogspot.com/ --------------------------------- -- Kernelnewbies: Help each other learn about the Linux kernel. Archive: http://mail.nl.linux.org/kernelnewbies/ FAQ: http://kernelnewbies.org/faq/
