On 05/01/2010 08:10 PM, Dan Magenheimer wrote:
Eventually you'll have to swap frontswap pages, or kill uncooperative
guests. At which point all of the simplicity is gone.
OK, now I think I see the crux of the disagreement.
Alas, I think we're pretty far from that.
NO! Frontswap on Xen+tmem never *never* _never_ NEVER results
in host swapping.
That's a bug. You're giving the guest memory without the means to take
it back. The result is that you have to _undercommit_ your memory
resources.
Consider a machine running a guest, with most of its memory free. You
give the memory via frontswap to the guest. The guest happily swaps to
frontswap, and uses the freed memory for something unswappable, like
mlock()ed memory or hugetlbfs.
Now the second node dies and you need memory to migrate your guests
into. But you can't, and the hypervisor is at the mercy of the guest
for getting its memory back; and the guest can't do it (at least not
quickly).
Host swapping is evil. Host swapping is
the root of most of the bad reputation that memory overcommit
has gotten from VMware customers. Host swapping can't be
avoided with some memory overcommit technologies (such as page
sharing), but frontswap on Xen+tmem CAN and DOES avoid it.
In this case the guest expects that swapped out memory will be slow
(since was freed via the swap API; it will be slow if the host happened
to run out of tmem). So by storing this memory on disk you aren't
reducing performance beyond what you promised to the guest.
Swapping guest RAM will indeed cause a performance hit, but sometimes
you need to do it.
So, to summarize:
1) You agreed that a synchronous interface for frontswap makes
sense for swap-to-in-kernel-compressed-RAM because it is
truly swapping to RAM.
Because the interface is internal to the kernel.
2) You have pointed out that an asynchronous interface for
frontswap makes more sense for KVM than a synchronous
interface, because KVM does host swapping.
kvm's host swapping is unrelated. Host swapping swaps guest-owned
memory; that's not what we want here. We want to cache guest swap in
RAM, and that's easily done by having a virtual disk cached in main
memory. We're simply presenting a disk with a large write-back cache to
the guest.
You could just as easily cache a block device in free RAM with Xen.
Have a tmem domain behave as the backend for your swap device. Use
ballooning to force tmem to disk, or to allow more cache when memory is
free.
Voila: you no longer depend on guests (you depend on the tmem domain,
but that's part of the host code), you don't need guest modifications,
so it works across a wider range of guests.
Then you said
if you have an asynchronous interface anyway, the existing
swap code works just fine with no changes so frontswap
is not needed at all... for KVM.
For any hypervisor which implements virtual disks with write-back cache
in host memory.
3) You have suggested that if Xen were more like KVM and required
host-swapping, then Xen doesn't need frontswap either.
Host swapping is not a requirement.
BUT frontswap on Xen+tmem always truly swaps to RAM.
AND that's a problem because it puts the hypervisor at the mercy of the
guest.
So there are two users of frontswap for which the synchronous
interface makes sense.
I believe there is only one. See below.
I believe there may be more in the
future and you disagree but, as Jeremy said, "a general Linux
principle is not to overdesign interfaces for hypothetical users,
only for real needs." We have demonstrated there is a need
with at least two users so the debate is only whether the
number of users is two or more than two.
Frontswap is a very non-invasive patch and is very cleanly
layered so that if it is not in the presence of either of
the intended "users", it can be turned off in many different
ways with zero overhead (CONFIG'ed off) or extremely small overhead
(frontswap_ops is never set; or frontswap_ops is set but the
underlying hypervisor doesn't support it so frontswap_poolid
never gets set).
The problem is not the complexity of the patch itself. It's the fact
that it introduces a new external API. If we refactor swapping, that
stands in the way.
How much, that's up to the mm maintainers to say. If it isn't a problem
for them, fine (but I still think
swap-to-RAM-without-hypervisor-decommit is a bad idea).
So... KVM doesn't need it and won't use it. Do you, Avi, have
any other objections as to why the frontswap patch shouldn't be
accepted as is for the users that DO need it and WILL use it?
Even ignoring the problems above (which are really hypervisor problems
and the guest, which is what we're discussing here, shouldn't care if
the hypervisor paints itself into an oom), a synchronous single-page DMA
API is a bad idea. Look at the Xen network and block code, while they
eventually do a memory copy for every page they see, they try to batch
multiple pages into an exit, and make the response asynchronous.
As an example, with a batched API you could save/restore the fpu context
and use sse for copying the memory, while with a single page API you'd
probably lost out. Synchronous DMA, even for emulated hardware, is out
of place in 2010.
--
error compiling committee.c: too many arguments to function
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@xxxxxxxxxx For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@xxxxxxxxx";> email@xxxxxxxxx </a>