2012/8/11 Jianpeng Ma <majianpeng@xxxxxxxxx>:
> On 2012-08-09 16:58 Shaohua Li <shli@xxxxxxxxxx> Wrote:
>>This is a new tempt to make raid5 handle stripes in multiple threads, as
>>suggested by Neil to have maxium flexibility and better numa binding. It
>>basically is a combination of my first and second generation patches. By
>>default, no multiple thread is enabled (all stripes are handled by raid5d).
>>
>>An example to enable multiple threads:
>>#echo 3 > /sys/block/md0/md/auxthread_number
>>This will create 3 auxiliary threads to handle stripes. The threads can run
>>on any cpus and handle stripes produced by any cpus.
>>
>>#echo 1-3 > /sys/block/md0/md/auxth0/cpulist
>>This will bind auxiliary thread 0 to cpu 1-3, and this thread will only handle
>>stripes produced by cpu 1-3. User tool can further change the thread's
>>affinity, but the thread can only handle stripes produced by cpu 1-3 till the
>>sysfs entry is changed again.
>>
>>If stripes produced by a CPU aren't handled by any auxiliary thread, such
>>stripes will be handled by raid5d. Otherwise, raid5d doesn't handle any
>>stripes.
>>
> I tested and found two problem(maybe not).
>
> 1:print cpulist of auxth, you maybe lost print the '\n'.
> diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
> index 7c8151a..3700cdc 100644
> --- a/drivers/md/raid5.c
> +++ b/drivers/md/raid5.c
> @@ -4911,9 +4911,13 @@ struct raid5_auxth_sysfs {
> static ssize_t raid5_show_thread_cpulist(struct mddev *mddev,
> struct raid5_auxth *thread, char *page)
> {
> + int n;
> if (!mddev->private)
> return 0;
> - return cpulist_scnprintf(page, PAGE_SIZE, &thread->work_mask);
> + n = cpulist_scnprintf(page, PAGE_SIZE - 2, &thread->work_mask);
> + page[n++] = '\n';
> + page[n] = 0;
> + return n;
> }
>
> static ssize_t
some sysfs entries print out '\n', some not, I don't mind add it
> 2: Test 'dd if=/dev/zero of=/dev/md0 bs=2M ', the performance regress remarkable.
> auxthread_number=0, 200MB/s;
> auxthread_number=4, 95MB/s.
So multiple threads handle stripes reduce request merge. In your
workload, raid5d isn't a bottleneck at all. In practice, I thought only
array which can drive high IOPS needs enable multi thread. And
if you create multiple threads, better let the threads handle different
cpus.
Thanks,
Shaohua
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
