Re: [PATCH][RF C/T/D] Unmapped page cache control - via boot parameter

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 03/17/2010 05:24 PM, Chris Webb wrote:
Avi Kivity<avi@xxxxxxxxxx>  writes:

On 03/15/2010 10:23 PM, Chris Webb wrote:

Wasteful duplication of page cache between guest and host notwithstanding,
turning on cache=writeback is a spectacular performance win for our guests.
Is this with qcow2, raw file, or direct volume access?
This is with direct access to logical volumes. No file systems or qcow2 in
the stack. Our typical host has a couple of SATA disks, combined in md
RAID1, chopped up into volumes with LVM2 (really just dm linear targets).
The performance measured outside qemu is excellent, inside qemu-kvm is fine
too until multiple guests are trying to access their drives at once, but
then everything starts to grind badly.


OK.

I can understand it for qcow2, but for direct volume access this
shouldn't happen.  The guest schedules as many writes as it can,
followed by a sync.  The host (and disk) can then reschedule them
whether they are in the writeback cache or in the block layer, and
must sync in the same way once completed.
I don't really understand what's going on here, but I wonder if the
underlying problem might be that all the O_DIRECT/O_SYNC writes from the
guests go down into the same block device at the bottom of the device mapper
stack, and thus can't be reordered with respect to one another.

They should be reorderable. Otherwise host filesystems on several volumes would suffer the same problems.

Whether the filesystem is in the host or guest shouldn't matter.

For our
purposes,

   Guest AA   Guest BB       Guest AA   Guest BB       Guest AA   Guest BB
   write A1                  write A1                             write B1
              write B1       write A2                  write A1
   write A2                             write B1       write A2

are all equivalent, but the system isn't allowed to reorder in this way
because there isn't a separate request queue for each logical volume, just
the one at the bottom. (I don't know whether nested request queues would
behave remotely reasonably either, though!)

Also, if my guest kernel issues (say) three small writes, one at the start
of the disk, one in the middle, one at the end, and then does a flush, can
virtio really express this as one non-contiguous O_DIRECT write (the three
components of which can be reordered by the elevator with respect to one
another) rather than three distinct O_DIRECT writes which can't be permuted?
Can qemu issue a write like that? cache=writeback + flush allows this to be
optimised by the block layer in the normal way.

Guest side virtio will send this as three requests followed by a flush. Qemu will issue these as three distinct requests and then flush. The requests are marked, as Christoph says, in a way that limits their reorderability, and perhaps if we fix these two problems performance will improve.

Something that comes to mind is merging of flush requests. If N guests issue one write and one flush each, we should issue N writes and just one flush - a flush for the disk applies to all volumes on that disk.

--
error compiling committee.c: too many arguments to function

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@xxxxxxxxxx  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@xxxxxxxxx";> email@xxxxxxxxx </a>

[Index of Archives]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Bugtraq]     [Linux]     [Linux OMAP]     [Linux MIPS]     [ECOS]     [Asterisk Internet PBX]     [Linux API]