> Von: linux-raid-owner@xxxxxxxxxxxxxxx [linux-raid-owner@xxxxxxxxxxxxxxx]" im Auftrag von "Shaohua Li [shli@xxxxxxxxxx]
> Gesendet: Donnerstag, 6. Oktober 2016 01:17
> An: Doug Dumitru
> Cc: linux-raid; gayatri.kammela@xxxxxxxxx; ravi.v.shankar@xxxxxxxxx; hpa@xxxxxxxxx; yu-cheng.yu@xxxxxxxxx; yuanhan.liu@xxxxxxxxx
> Betreff: Re: Prefetch in /lib/raid6/avx2.c
>
> On Sun, Oct 02, 2016 at 03:40:09PM -0700, Doug Dumitru wrote:
> > I have been doing some high bandwidth testing of raid-6, and the
> > pretetch in raid6_avx24_gen_syndrome appears to be less than optimal.
> >
> > This is my patch (against 4.4.0-38 [Ubuntu 16.04LTS)
> >
> > --- cut here ---
> > --- lib/raid6/avx2.c0 2016-10-01 21:42:25.280347868 -0700
> > +++ lib/raid6/avx2.c 2016-10-02 15:35:48.168480760 -0700
> > @@ -189,10 +189,8 @@
> >
> > for (z = z0; z >= 0; z--) {
> >
> > - asm volatile("prefetchnta %0" : : "m" (dptr[z][d]));
> > - asm volatile("prefetchnta %0" : : "m" (dptr[z][d+32]));
> > - asm volatile("prefetchnta %0" : : "m" (dptr[z][d+64]));
> > - asm volatile("prefetchnta %0" : : "m" (dptr[z][d+96]));
> > + asm volatile("prefetchnta %0" : : "m" (dptr[z][d+128]));
> > + asm volatile("prefetchnta %0" : : "m" (dptr[z][d+192]));
>From the first look that looks strange.
1) It will add 2 prefetches for the last blocks beyond the data. Feels like bad coding.
2) The prefetch for the next block is already in the next loop (d+128)
Maybe the prefetcher takes longer than expected. And thus the next loop
will benefit from the "relocated" hint.
> >
> > asm volatile("vpcmpgtb %ymm4,%ymm1,%ymm5");
> > asm volatile("vpcmpgtb %ymm6,%ymm1,%ymm7");
> > --- cut here ---
> >
> > In perf, the cpu cycles goes from 5.3% to 3.0% for
> > raid6_avx24_gen_syndrome in my test and throughput increases from
> > about 8.2GB/sec to almost 10GB/sec. It is a very "synthetic" test,
> > but the avx2 code does seem to be a factor.
> >
> > I suspect other SSE and AVX "unroll variants" have similar issues, but
> > I have not tested those.
> >
> > My test system is an E5-1650 v3 (single socket) with DDR4. This might
> > help dual sockets even more.
>
> CC some intel folks to see if they have ideas
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@xxxxxxxxxxxxxxx
> More majordomo info at http://vger.kernel.org/majordomo-info.html
****************************************************************************
Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte
Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail
irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und
vernichten Sie diese Mail. Das unerlaubte Kopieren sowie die unbefugte
Weitergabe dieser Mail ist nicht gestattet.
�ber das Internet versandte E-Mails können unter fremden Namen erstellt oder
manipuliert werden. Deshalb ist diese als E-Mail verschickte Nachricht keine
rechtsverbindliche Willenserklärung.
Collogia
Unternehmensberatung AG
Ubierring 11
D-50678 Köln
Vorstand:
Kadir Akin
Dr. Michael Höhnerbach
Vorsitzender des Aufsichtsrates:
Hans Kristian Langva
Registergericht: Amtsgericht Köln
Registernummer: HRB 52 497
This e-mail may contain confidential and/or privileged information. If you
are not the intended recipient (or have received this e-mail in error)
please notify the sender immediately and destroy this e-mail. Any
unauthorized copying, disclosure or distribution of the material in this
e-mail is strictly forbidden.
e-mails sent over the internet may have been written under a wrong name or
been manipulated. That is why this message sent as an e-mail is not a
legally binding declaration of intention.
Collogia
Unternehmensberatung AG
Ubierring 11
D-50678 Köln
executive board:
Kadir Akin
Dr. Michael Höhnerbach
President of the supervisory board:
Hans Kristian Langva
Registry office: district court Cologne
Register number: HRB 52 497
****************************************************************************
