On Thu, Nov 11, 2010 at 8:22 AM, Sebastian Kayser <sebastian@xxxxxxxxxx> wrote:
> * Sebastian Kayser <sebastian@xxxxxxxxxx> wrote:
>> Just to make sure my understanding is correct:
>> - direct=1 should mitigate (disable?) OS caching effects
>> - sync=1, iodepth=1 should make sure that an I/O has really made it to
>> disk before the next on is issued, i.e. should de-facto disable
>> I/O coalescing or device caching
>>
>> Are these sane/valid assumptions?
>
> Shawn Lewis sent me an email off-list and suggested that the disk itself
> will still do write-caching (despite O_SYNC) unless this has been
> explicitly disabled. I dug into the storage configuration and found an
> option which sounded promising ("Drive Delayed Write", not to be confused
> with the RAID controller write cache which was disabled from the start).
> Disabled the delayed writes and the IOPS went down from ~240 to
>
> * ~100 for a filesystem test (size=100g out of 2TB)
> * ~110 for a raw device test
>
> Still a bit more than I would expect [1], but definitely closer. I am not
> sure whether this can be brought down any further, i.e. whether there's
> another caching effect hidden somwhere or if these figures now actually
> reflect the "raw" harddisk performance.
Sorry, I didn't mean to send that off list. There shouldn't be any other caches.
Did any of your tests use the full device or were you still setting
the size= parameter? Set size=2T or get rid of the size= parameter
entirely for the raw device test and you should see more like 70 or 80
iops.
I believe part of the reason the gain is so huge with the drive cache
is that you were also only using the outer diameter of the disk. If
you use the full disk but enable drive cache I think you'll see about
140 or 150iops.
>
> Either way, I am surprised how much gain in IOPS a simple hard drive cache
> delivers ... and I am also very grateful for all the suggestions so far!
> Thanks guys!
>
> Sebastian
>
> [1] http://www.spinics.net/lists/fio/msg00558.html
> --
> To unsubscribe from this list: send the line "unsubscribe fio" in
> the body of a message to majordomo@xxxxxxxxxxxxxxx
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe fio" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
