Okay, I almost have it! I've added a configure rule to detect whether yasm is happy, and fixed a few things in the wrapper and makefile. https://github.com/ceph/ceph/commits/wip-ec-isa https://github.com/ceph/ceph/pull/2180 The remaining issue is that yasm 1.1 can't seem to assemble several files. For example, $ yasm -f elf64 -i src/ceph/src/erasure-code/isa/isa-l/include/ src/erasure-code/isa/isa-l/erasure_code/gf_vect_dot_prod_avx2.asm.s -o /dev/null src/erasure-code/isa/isa-l/erasure_code/gf_vect_dot_prod_avx2.asm.s:133: error: undefined symbol `gf_vect_dot_prod_avx2.return_fail' (first use) src/erasure-code/isa/isa-l/erasure_code/gf_vect_dot_prod_avx2.asm.s:133: error: (Each undefined symbol is reported only once.) src/erasure-code/isa/isa-l/erasure_code/gf_vect_dot_prod_avx2.asm.s:137: error: instruction expected after label src/erasure-code/isa/isa-l/erasure_code/gf_vect_dot_prod_avx2.asm.s:149: error: instruction expected after label src/erasure-code/isa/isa-l/erasure_code/gf_vect_dot_prod_avx2.asm.s:150: error: instruction expected after label src/erasure-code/isa/isa-l/erasure_code/gf_vect_dot_prod_avx2.asm.s:165: error: undefined symbol `vperm2i128.next_vect' (first use) src/erasure-code/isa/isa-l/erasure_code/gf_vect_dot_prod_avx2.asm.s:170: error: undefined symbol `vperm2i128.loop32' (first use) Paul, is this expected? For now I am just detecting this in configure and skipping the plugin if it fails, but it would be nice if this worked on ubuntu 12.04 and el6. sage On Thu, 31 Jul 2014, Andreas Joachim Peters wrote: > Ok, > > I fixed the 'make dist' problem and hopefully also the YASM include problem. > > The CEPH yasm-wrapper was removing '-I' from the options passed to yasm, so '-I' didn't work without patching the wrapper itself. > > I looked into YASM 1.1 and I found that at a certain patch version they fixed the behaviour for relative -I paths, so I use now "-I $(abs_srcdir)" hoping that this works with both YASM versions. > > Let me know, > thanks Andreas. > > ________________________________________ > From: ceph-devel-owner@xxxxxxxxxxxxxxx [ceph-devel-owner@xxxxxxxxxxxxxxx] on behalf of Andreas Joachim Peters [Andreas.Joachim.Peters@xxxxxxx] > Sent: 31 July 2014 09:39 > To: Sage Weil; Ma, Jianpeng > Cc: ceph-devel@xxxxxxxxxxxxxxx > Subject: RE: Pull Request for ISA EC plug-in > > Ok, > thanks for looking into that. > > I am fixing the 'make dist' target and will try to make it work also with yasm 1.1 ... > > Will let you know when to retry ... > > Andreas. > > ________________________________________ > From: ceph-devel-owner@xxxxxxxxxxxxxxx [ceph-devel-owner@xxxxxxxxxxxxxxx] on behalf of Sage Weil [sweil@xxxxxxxxxx] > Sent: 31 July 2014 07:27 > To: Ma, Jianpeng > Cc: Andreas Joachim Peters; ceph-devel@xxxxxxxxxxxxxxx > Subject: RE: Pull Request for ISA EC plug-in > > On Thu, 31 Jul 2014, Ma, Jianpeng wrote: > > Hi, > > At my machine, I also met this bug. But I modify this, it can work. > > diff --git a/src/erasure-code/isa/Makefile.am b/src/erasure-code/isa/Makefile.am > > index eebffea..dee593b 100644 > > --- a/src/erasure-code/isa/Makefile.am > > +++ b/src/erasure-code/isa/Makefile.am > > @@ -34,7 +34,7 @@ libec_isa_la_SOURCES = ${isa_sources} > > > > libec_isa_la_CFLAGS = ${AM_CFLAGS} -I $(srcdir)/erasure-code/isa/isa-l/include/ > > libec_isa_la_CXXFLAGS = ${AM_CXXFLAGS} -I $(srcdir)/erasure-code/isa/isa-l/include/ > > -libec_isa_la_CCASFLAGS = ${AM_CCASFLAGS} -I $(srcdir)/erasure-code/isa/isa-l/include/ > > +libec_isa_la_CCASFLAGS = ${AM_CCASFLAGS} -i $(srcdir)/erasure-code/isa/isa-l/include > > > > libec_isa_la_LIBADD = $(LIBCRUSH) $(PTHREAD_LIBS) $(EXTRALIBS) > > libec_isa_la_LDFLAGS = ${AM_LDFLAGS} -version-info 2:10:0 > > > > > > As Andreas said, the different is -I and -i. > > But how to choose? My yasm version is 1.2.0. > > Ah, my machine with 1.2 is behaving. I think it's a combination of that, > and that the files are not all include in the 'make dist' tarball. > > sage > > > > > > > Thanks! > > > > > > > > > -----Original Message----- > > > From: ceph-devel-owner@xxxxxxxxxxxxxxx > > > [mailto:ceph-devel-owner@xxxxxxxxxxxxxxx] On Behalf Of Sage Weil > > > Sent: Thursday, July 31, 2014 7:02 AM > > > To: Andreas Joachim Peters > > > Cc: ceph-devel@xxxxxxxxxxxxxxx > > > Subject: RE: Pull Request for ISA EC plug-in > > > > > > On Wed, 30 Jul 2014, Andreas Joachim Peters wrote: > > > > Hi Sage, > > > > > > > > my guess it comes from the yasm include option which evt. is > > > > non-standard. Mine documents '-i' and '-I' while I found on the web > > > > that probably only '-I' is the portable one. > > > > > > > > I have changed that, still I can only guess, since I cannot reproduce. > > > > > > Hmm, how are you building it? Running make on my work box (precise) also > > > fails. I modified yasm-wrapper slightly[1] and it's gets further, but still no > > > dice... > > > > > > Thanks! > > > sage > > > > > > > > > [1] https://github.com/ceph/ceph/commit/wip-ec-isa > > > > > > > > > > > > > > Please rerun. > > > > > > > > Thanks, Andreas. > > > > > > > > ________________________________________ > > > > From: Sage Weil [sweil@xxxxxxxxxx] > > > > Sent: 30 July 2014 19:46 > > > > To: Andreas Joachim Peters > > > > Cc: ceph-devel@xxxxxxxxxxxxxxx > > > > Subject: RE: Pull Request for ISA EC plug-in > > > > > > > > On Mon, 28 Jul 2014, Andreas Joachim Peters wrote: > > > > > Hi Sage, > > > > > > > > > > I fixed that. I missed '$(srcdir)' in the assembler and C/C++ > > > > > include statements (I always compiled in src dir ..) > > > > > > > > > > You can retry. > > > > > > > > I think there are still a few issues: > > > > > > > > make[4]: *** > > > > [erasure-code/isa/isa-l/erasure_code/libec_isa_la-ec_base.lo] Error 1 > > > > make[4]: *** Waiting for unfinished jobs.... > > > > ./yasm-wrapper: yasm -f elf64 -i ./erasure-code/isa/isa-l/include/ > > > > erasure-code/isa/isa-l/erasure_code/ec_multibinary.asm.s -o > > > > erasure-code/isa/isa-l/erasure_code/.libs/libec_isa_la-ec_multibinary. > > > > asm.o > > > > FATAL: yasm: unable to open include file `reg_sizes.asm' > > > > ./yasm-wrapper: yasm -f elf64 -i ./erasure-code/isa/isa-l/include/ > > > > erasure-code/isa/isa-l/erasure_code/gf_2vect_dot_prod_avx2.asm.s -o > > > > erasure-code/isa/isa-l/erasure_code/.libs/libec_isa_la-gf_2vect_dot_pr > > > > od_avx2.asm.o > > > > error: > > > > erasure-code/isa/isa-l/erasure_code/gf_2vect_dot_prod_avx2.asm.s:157: > > > > undefined symbol `gf_2vect_dot_prod_avx2.return_fail' (first use) > > > > error: > > > > erasure-code/isa/isa-l/erasure_code/gf_2vect_dot_prod_avx2.asm.s:157: > > > > (Each undefined symbol is reported only once.) > > > > error: > > > > erasure-code/isa/isa-l/erasure_code/gf_2vect_dot_prod_avx2.asm.s:161: > > > > instruction expected after label > > > > error: > > > > erasure-code/isa/isa-l/erasure_code/gf_2vect_dot_prod_avx2.asm.s:178: > > > > instruction expected after label > > > > error: > > > > erasure-code/isa/isa-l/erasure_code/gf_2vect_dot_prod_avx2.asm.s:179: > > > > instruction expected after label > > > > error: > > > > erasure-code/isa/isa-l/erasure_code/gf_2vect_dot_prod_avx2.asm.s:183: > > > > instruction expected after label > > > > error: > > > > erasure-code/isa/isa-l/erasure_code/gf_2vect_dot_prod_avx2.asm.s:184: > > > > instruction expected after label > > > > error: > > > > erasure-code/isa/isa-l/erasure_code/gf_2vect_dot_prod_avx2.asm.s:206: > > > > undefined symbol `vperm2i128.next_vect' (first use) > > > > error: > > > > erasure-code/isa/isa-l/erasure_code/gf_2vect_dot_prod_avx2.asm.s:213: > > > > undefined symbol `vperm2i128.loop32' (first use) > > > > make[4]: *** > > > > [erasure-code/isa/isa-l/erasure_code/libec_isa_la-ec_multibinary.asm.l > > > > o] Error 1 > > > > make[4]: *** > > > > [erasure-code/isa/isa-l/erasure_code/libec_isa_la-gf_2vect_dot_prod_av > > > > x2.asm.lo] Error 1 > > > > ./yasm-wrapper: yasm -f elf64 -i ./erasure-code/isa/isa-l/include/ > > > > erasure-code/isa/isa-l/erasure_code/gf_2vect_dot_prod_avx.asm.s -o > > > > erasure-code/isa/isa-l/erasure_code/.libs/libec_isa_la-gf_2vect_dot_pr > > > > od_avx.asm.o In file included from > > > > erasure-code/isa/isa-l/erasure_code/ec_highlevel_func.c:30:0: > > > > ./erasure-code/isa/isa-l/include/erasure_code.h:52:25: fatal error: > > > > gf_vect_mul.h: No such file or directory compilation terminated. > > > > > > > > This is from > > > > > > > http://gitbuilder.sepia.ceph.com/gitbuilder-ceph-deb-precise-amd64-basic/log. > > > cgi?log=c7e765e22427958c0d03fbfec42a29aa34895ef3. > > > > > > > > Thanks! > > > > sage > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks Andreas. > > > > > > > > > > ________________________________________ > > > > > From: ceph-devel-owner@xxxxxxxxxxxxxxx > > > > > [ceph-devel-owner@xxxxxxxxxxxxxxx] on behalf of Sage Weil > > > > > [sweil@xxxxxxxxxx] > > > > > Sent: 29 July 2014 00:22 > > > > > To: Andreas Joachim Peters > > > > > Cc: ceph-devel@xxxxxxxxxxxxxxx > > > > > Subject: Re: Pull Request for ISA EC plug-in > > > > > > > > > > Hi Andreas! > > > > > > > > > > On Mon, 28 Jul 2014, Andreas Joachim Peters wrote: > > > > > > Hi all, > > > > > > > > > > > > here is a PULL request for the ISA EC plugin rebased against > > > > > > master of today for review. > > > > > > > > > > > > https://github.com/ceph/ceph/pull/2155 > > > > > > > > > > > > I have added as discussed the exhaustive test of all possible > > > > > > failures scenarios for both supported matrix types for a > > > > > > (k=12,m=4) configuration to the unit tests ( which helped indeed > > > > > > to spot a bug in the table caching ) > > > > > > > > > > I tried to push this through the build system and it looks like > > > > > perhaps some files are missing from the commit? > > > > > > > > > > CC > > > > > erasure-code/isa/isa-l/erasure_code/libec_isa_la-ec_highlevel_func.l > > > > > o In file included from > > > > > erasure-code/isa/isa-l/erasure_code/ec_base.c:32:0: > > > > > erasure-code/isa/isa-l/include/erasure_code.h:52:25: fatal error: > > > > > gf_vect_mul.h: No such file or directory compilation terminated. > > > > > CCAS > > > > > erasure-code/isa/isa-l/erasure_code/libec_isa_la-ec_multibinary.asm. > > > > > lo CCAS > > > > > erasure-code/isa/isa-l/erasure_code/libec_isa_la-gf_2vect_dot_prod_a > > > > > vx2.asm.lo In file included from > > > > > erasure-code/isa/isa-l/erasure_code/ec_highlevel_func.c:30:0: > > > > > erasure-code/isa/isa-l/include/erasure_code.h:52:25: fatal error: > > > > > gf_vect_mul.h: No such file or directory compilation terminated. > > > > > > > > > > and > > > > > > > > > > FATAL: yasm: unable to open include file `reg_sizes.asm' > > > > > > > > > > See > > > > > > > > > > http://gitbuilder.sepia.ceph.com/gitbuilder-ceph-deb-wheezy-amd64-ba > > > > > sic/log.cgi?log=60408e6c6b3b34d719943d16fcc023407e08ae01 > > > > > > > > > > Hopefully these are just sitting in your working directory? > > > > > > > > > > Thanks! > > > > > sage > > > > > -- > > > > > To unsubscribe from this list: send the line "unsubscribe > > > > > ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx > > > > > More majordomo info at http://vger.kernel.org/majordomo-info.html > > > > > -- > > > > > To unsubscribe from this list: send the line "unsubscribe > > > > > ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx > > > > > More majordomo info at http://vger.kernel.org/majordomo-info.html > > > > > > > > > > > > > > -- > > > > To unsubscribe from this list: send the line "unsubscribe ceph-devel" > > > > in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo > > > > info at http://vger.kernel.org/majordomo-info.html > > > > > > > > > > > -- > > > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body > > > of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at > > > http://vger.kernel.org/majordomo-info.html > > -- > > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > > the body of a message to majordomo@xxxxxxxxxxxxxxx > > More majordomo info at http://vger.kernel.org/majordomo-info.html > > > > > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html > > -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
