Hello,
Doing some benchmark tests for my patch series I discovered that
direct I/O performance for a single thread onto raid456 md seems
to be not on par with what I would have expected.
A simple 5 minute test run with md on top of /dev/ramX ramdisks
reveals:
raid5 - 3 disks - chunk 32k - 1 writer
4K random direct IO writes: 5MB/sec (five megabytes per second)
raid5 - 3 disks - chunk 32k - 1 writer
4K random direct IO writes: 600MB/sec
raid5 - 3 disks - chunk 32k - 8 writers
4K random direct IO writes: 45MB/sec
raid5 - 3 disks - chunk 32k - 8 writers
4K random direct IO writes: 800MB/sec
for comparison:
raid1 - 3 disks - 1 writer
4K random direct IO writes: 150MB/sec
That would be a 1ms penalty per one md block write (or to be
precise per 3 device I/Os). A similar I/O overhead on the three
disk RAID1 shows reasonable numbers.
A perf record of the 1 writer test gives:
38.40% swapper [kernel.kallsyms] [k] default_idle
13.14% md0_raid5 [kernel.kallsyms] [k] _raw_spin_unlock_irqrestore
13.05% swapper [kernel.kallsyms] [k] tick_nohz_idle_enter
10.01% iot [raid456] [k] raid5_unplug
9.06% swapper [kernel.kallsyms] [k] tick_nohz_idle_exit
3.39% md0_raid5 [kernel.kallsyms] [k] __kernel_fpu_begin
1.67% md0_raid5 [xor] [k] xor_sse_2_pf64
0.87% iot [kernel.kallsyms] [k] finish_task_switch
I'm confused and clueless. Especially I cannot see where the
10% overhead in the source of raid5_unplug might come
from? Any idea from someone with better insight?
Markus
****************************************************************************
Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte
Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail
irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und
vernichten Sie diese Mail. Das unerlaubte Kopieren sowie die unbefugte
Weitergabe dieser Mail ist nicht gestattet.
�ber das Internet versandte E-Mails können unter fremden Namen erstellt oder
manipuliert werden. Deshalb ist diese als E-Mail verschickte Nachricht keine
rechtsverbindliche Willenserklärung.
Collogia
Unternehmensberatung AG
Ubierring 11
D-50678 Köln
Vorstand:
Kadir Akin
Dr. Michael Höhnerbach
Vorsitzender des Aufsichtsrates:
Hans Kristian Langva
Registergericht: Amtsgericht Köln
Registernummer: HRB 52 497
This e-mail may contain confidential and/or privileged information. If you
are not the intended recipient (or have received this e-mail in error)
please notify the sender immediately and destroy this e-mail. Any
unauthorized copying, disclosure or distribution of the material in this
e-mail is strictly forbidden.
e-mails sent over the internet may have been written under a wrong name or
been manipulated. That is why this message sent as an e-mail is not a
legally binding declaration of intention.
Collogia
Unternehmensberatung AG
Ubierring 11
D-50678 Köln
executive board:
Kadir Akin
Dr. Michael Höhnerbach
President of the supervisory board:
Hans Kristian Langva
Registry office: district court Cologne
Register number: HRB 52 497
****************************************************************************
