Thank you Izik for the sharing. On Mon, Aug 31, 2009 at 12:55 AM, Izik Eidus<ieidus@xxxxxxxxxx> wrote: > Peter Teoh wrote: >> >> Thank you Shailesh for the comments, >> >> On Sun, Aug 30, 2009 at 4:10 AM, shailesh >> jain<coolworldofshail@xxxxxxxxx> wrote: >> >>> >>> On Sun, Aug 30, 2009 at 3:43 AM, Peter Teoh <htmldeveloper@xxxxxxxxx> >>> wrote: >>> >>>> >>>> Thank you Minchan. You made me curious now...:-). >>>> >>>> On Sat, Aug 29, 2009 at 9:37 PM, Minchan Kim<minchan.kim@xxxxxxxxx> >>>> wrote: >>>> >>>>> >>>>> Hi, Peter. >>>>> >>>>> On Sun, Aug 30, 2009 at 9:20 AM, Peter Teoh<htmldeveloper@xxxxxxxxx> >>>>> wrote: >>>>> >>>>>> >>>>>> The papers are located here: >>>>>> >>>>>> http://www.linuxsymposium.org/2009/ls-2009-proceedings.pdf >>>>>> >>>>>> I have not read any papers yet. But given limited amount of time - >>>>>> which papers do u think are the interesting one to read? And perhaps >>>>>> >>>>> >>>>> In my case, I will take a "Increasing memory density by using KSM". >>>>> >>>>> >>>> >>>> Have not really pursued deeply other than: >>>> >>>> http://lkml.org/lkml/2009/4/19/210 >>>> >>>> But conceptually: >>>> >>>> a. what is the criteria of deciding when two piece of memory are >>>> identical? (if it is byte-for-byte comparison then performance will >>>> be seriously disastrous?) >>>> >>> >>> you dont need to do that everytime ... think hashes... >>> and only if they match go ahead compare byte-by-byte >>> >>> >> >> after reading: >> >> http://lkml.org/lkml/2009/4/19/215 >> >> I came to the suspicion that it is talking about userspace memory >> pages, not kernel pages. Izik, correct? >> > > Ksm scan userspace anonymous memory, from the kvm point of view this is the > whole memory of the guest (include the guest kernel and userspace). > >> hashing these pages - even though it is only periodically - will still >> incur a lot of calculation: if n pages belong to KVM1, and m pages >> belong to KVM2, then altogether there will be n * m pages of >> comparison. > > Wrong, ksm, use sorted tree to search pages, the result is log(n+m) (this > number is little bit different, but should give perspective) in worst case, > and we are not hasing the pages all the time. > (Check for sorted trees algorithems to read about it) > >> And don't forget that when u are reading from any pages, >> the entire KVMx must be frozen from modifying the memory. > > This is not true, ksm first find two identical pages, then only when it > find, it mark the page as readonly and then recompare it, this doesnt effect > interrupt latency. > >> ie, >> interrupt latency involved. and worst, different CPU may be running >> the same KVMx in hypervisor or non-hypervisor mode. but to ensure >> that the memory being read is not undergoing changes, i think u really >> have to freeze all the CPU running the same KVMx. > > Not true, check above. > >> logically if all >> the CPU are running the same KVMx in hypervisor mode, the VM client >> memory is effectively frozen....but if any of them is not.....u have >> to trigger a VMEXIT condition. >> >> another problem is that the different KVM may be running different set >> of userspace applications at the same time.....so the proportion of >> pages that can be identical to each other is a very small nos....it >> really defeats all the efforts in memory scanning and matching >> identical pages. >> > > Most of the identical pages are kernel libaries and system cache. > >> but I do have some optimistic comment though: >> >> in its current design, it is randomly picking pages to do the >> identical pages comparison....and doing it periodically >> ......alternative to that is that it should allow the administrator to >> specify which pages are set to be identical for all the OS - and this >> entirely avoided the need for recalculating the hashes for comparison >> altogether. ie, once shared, it is permanent shared.....and only one >> copy of it exists in the system - to be used for all KVM. >> >> Since all KVM are identically running the same image, then this could >> be specified SOMEWHERE at the KVM client's vmlinux level.....(even >> that there are many area within the vmlinux that are read-writeable, >> so questions is where are the identical pages coming from?)...but then >> again....kernel image's memory management is much worst than userspace >> memory management.....i cannot imagine the complexity of doing >> that...... >> >> if there are sufficiently large numbers of KVM ....this could bring >> huge benefits in terms of memory saving....comments? >> >> >>>> >>>> b. and after memory memory coalescing....due to "identical" memory - >>>> necessarily this is applicable to read-only memory right? (if not >>>> then it may be modified later....and u need to trigger copy-on-write >>>> mechanism to duplicate the memory piece....lots of overheads i >>>> supposed). >>>> >>> >>> cow it is. >>> >>> >>>> >>>> c. but isn't it very dangerous - even though it is read-only....one >>>> KVM client can actually modify the memory (just set_memory_rw() to >>>> >> >> to answer my own question.....my earlier assumption was that KSM is >> doing identity-comparison on kernel memory...which is not i think. >> not sure if Izik can comment? >> > > KVM by itself is secure and trusted enviorment from the kernel point of > view, if kvm mark the guest memory as readonly the guest cant change it > without kvm will know about it (and in this case will break the shared page) > Can you share more KVM's pagetable management? For Xen, I know the hypervisor set all the physical mem as readonly. And the VM client (or Xen client) have its own view (pagetable) of the read-write attribute of the memory allocated to it. And everytime VM client tried to write to memory, it will trigger a VMEXIT condition, and in the hypervisor mode, the memory will then be toggled to read-write, and memory updated, and then toggled back to read-only again. So hypervisor have total control of read-writeability of memory in a sense. And thus necessarily implement a separate pagetable (shadow pagetable) for describing these attributes. But then in the above scenario, it is also quite impossible for the Xen server to know which part of memory belongs (possible, but tedious) to the vmlinux image of the Xen client (which is readonly) and which part belong to the heap memory (read-writeable).....so effectively the Xen client CAN always modify its own vmlinux executable image (eg, malware WILL do that) and that will result in two Xen client running identical OS but ended having different memory pages. This was what I meant originally - different clients is able to modify its own kernel memory......just by using set_memory_rw() API. Another scenario possible is a worm that propagate from corrupted memory to all other non-corrupted memory.....but that I will detail after confirmation of my understanding (above) for KVM pagetable management first. Thank you for the sharing :-). >> >>>> >>>> turn it read-write, and then write to it.....and effectly the entire >>>> chain of KVM effectively is compromised via having a corrupted piece >>>> of memory? but then again i am suspecting i am completely >>>> wrong.....as logically KVM client should not be able to change >>>> memory's PTE attribute just purely using set_memory_rw() as a VM >>>> client. if this is true, then it means that u need duplication of >>>> the pagetables to describes the memory attributes - one for the VM >>>> client, and one for the VM host? Sorry....completely newbie... >>>> >>>> d. on the other hand....if it is really permanently set to >>>> readonly....and a KVM client after calling set_memory_rw()...still >>>> will NOT be able to set the memory read-write....then immediately the >>>> KVM client can deduce something about the server....eg....VM >>>> environment...as real x86 hardware does not have memory readonly >>>> setting capability... >>>> >>>> >>>>>> >>>>>> one-liners comment on why? >>>>>> >>>>> >>>>> There is no special reason. If I have to say, >>>>> >>>>> 1. I have a interesting in mm, >>>>> 2, KSM is one of hot topics these days. >>>>> 3. KSM can affect both server and embedded device >>>>> 4. I know authour's many contiribution in linux kernel so that I don't >>>>> doubt paper's quality. >>>>> >>>>> :) >>>>> >>>> >>>> Thank you for the sharing.... >>>> >> >> >> >> > > -- Regards, Peter Teoh -- To unsubscribe from this list: send an email with "unsubscribe kernelnewbies" to ecartis@xxxxxxxxxxxx Please read the FAQ at http://kernelnewbies.org/FAQ
