On Sat, Mar 11, 2023 at 12:40 AM Conor Dooley <conor@xxxxxxxxxx> wrote:
>
> On Fri, Mar 10, 2023 at 11:35:57PM +0800, Bin Meng wrote:
> > On Fri, Mar 10, 2023 at 11:07 PM Bin Meng <bmeng.cn@xxxxxxxxx> wrote:
> > >
> > > > From: Conor Dooley <conor.dooley@xxxxxxxxxxxxx>
> > > >
> > > > The spec folk, in their infinite wisdom, moved both control and status
> > > > registers & the FENCE.I instructions out of the I extension into their
> > > > own extensions (Zicsr, Zifencei) in the 20190608 version of the ISA
> > > > spec [0].
> > > > The GCC/binutils crew decided [1] to move their default version of the
> > > > ISA spec to the 20191213 version of the ISA spec, which came into being
> > > > for version 2.38 of binutils and GCC 12. Building with this toolchain
> > > > configuration would result in assembler issues:
> > > > CC arch/riscv/kernel/vdso/vgettimeofday.o
> > > > <<BUILDDIR>>/arch/riscv/include/asm/vdso/gettimeofday.h: Assembler messages:
> > > > <<BUILDDIR>>/arch/riscv/include/asm/vdso/gettimeofday.h:71: Error: unrecognized opcode `csrr a5,0xc01'
> > > > <<BUILDDIR>>/arch/riscv/include/asm/vdso/gettimeofday.h:71: Error: unrecognized opcode `csrr a5,0xc01'
> > > > <<BUILDDIR>>/arch/riscv/include/asm/vdso/gettimeofday.h:71: Error: unrecognized opcode `csrr a5,0xc01'
> > > > <<BUILDDIR>>/arch/riscv/include/asm/vdso/gettimeofday.h:71: Error: unrecognized opcode `csrr a5,0xc01'
> > > > This was fixed in commit 6df2a016c0c8 ("riscv: fix build with binutils
> > > > 2.38") by Aurelien Jarno, but has proven fragile.
> > > >
> > > > Before LLVM 17, LLVM did not support these extensions and, as such, the
> > > > cc-option check added by Aurelien worked. Since commit 22e199e6afb1
> > > > ("[RISCV] Accept zicsr and zifencei command line options") however, LLVM
> > > > *does* support them and the cc-option check passes.
> > > >
> > > > This surfaced as a problem while building the 5.10 stable kernel using
> > > > the default Tuxmake Debian image [2], as 5.10 did not yet support ld.lld,
> > > > and uses the Debian provided binutils 2.35.
> > > > Versions of ld prior to 2.38 will refuse to link if they encounter
> > > > unknown ISA extensions, and unfortunately Zifencei is not supported by
> > > > bintuils 2.35.
> > > >
> > > > Instead of dancing around with adding these extensions to march, as we
> > > > currently do, Palmer suggested locking GCC builds to the same version of
> > > > the ISA spec that is used by LLVM. As far as I can tell, that is 2.2,
> > > > with, apparently [3], a lack of interest in implementing a flag like
> > > > GCC's -misa-spec at present.
> > > >
> > > > Add {cc,as}-option checks to add -misa-spec to KBUILD_{A,C}FLAGS when
> > > > GCC is used & remove the march dance.
> > > >
> > > > As clang does not accept this argument, I had expected to encounter
> > > > issues with the assembler, as neither zicsr nor zifencei are present in
> > > > the ISA string and the spec version *should* be defaulting to a version
> > > > that requires them to be present. The build passed however and the
> > > > resulting kernel worked perfectly fine for me on a PolarFire SoC...
> > > >
> > > > Link: https://riscv.org/wp-content/uploads/2019/06/riscv-spec.pdf [0]
> > > > Link: https://groups.google.com/a/groups.riscv.org/g/sw-dev/c/aE1ZeHHCYf4 [1]
> > > > Link: https://lore.kernel.org/all/CA+G9fYt9T=ELCLaB9byxaLW2Qf4pZcDO=huCA0D8ug2V2+irJQ@xxxxxxxxxxxxxx/ [2]
> > > > Link: https://discourse.llvm.org/t/specifying-unpriviledge-spec-version-misa-spec-gcc-flag-equivalent/66935 [3]
> > > > CC: stable@xxxxxxxxxxxxxxx
> > > > Suggested-by: Palmer Dabbelt <palmer@xxxxxxxxxxxx>
> > > > Reported-by: Naresh Kamboju <naresh.kamboju@xxxxxxxxxx>
> > > > Signed-off-by: Conor Dooley <conor.dooley@xxxxxxxxxxxxx>
> > > > ---
> > > > I think Aurelien's original commit message might actually not be quite
> > > > correct? I found, in my limited testing, that it is not the default
> > > > behaviour of gas that matters, but rather the toolchain itself?
> > > > My binutils versions (both those built using the clang-built-linux
> > > > tc-build scripts which do not set an ISA spec version, and one built
> > > > using the riscv-gnu-toolchain infra w/ an explicit 20191213 spec version
> > > > set) do not encounter these issues.
> > >
> > > I am unable to reproduce the build failure as reported by commit 6df2a016c0c8
> > > ("riscv: fix build with binutils 2.38") by Aurelien Jarno using kernel.org
> > > pre-built GCC 11.3.0 [1] which includes binutils 2.38.
> > >
> > > [1] https://mirrors.edge.kernel.org/pub/tools/crosstool/files/bin/x86_64/11.3.0/x86_64-gcc-11.3.0-nolibc-x86_64-linux.tar.xz
> > >
> > > The defconfig of v5.16 kernel (commit 6df2a016c0c8 lands in v5.17) builds fine
> > > for me. Anything I am missing?
> > >
> >
> > Some further note:
> >
> > After I switched to kernel.org pre-built GCC 12.2.0 [1] which includes
> > binutils 2.39, I was able to reproduce the exact same build failure of
> > v5.16 kernel as described in the commit 6df2a016c0c8 ("riscv: fix
> > build with binutils 2.38") by Aurelien Jarno.
> >
> > To verify the commit message of 6df2a016c0c8 is accurate or not, I
> > built a GAS from binutils 2.37 and replaced the GAS 2.39 in the
> > kernel.org package, surprisingly kernel v5.16 did not build with the
> > same build failure.
> >
> > So it seems that it's GCC that caused the build failure instead of GAS
> > from binutils??
>
> Right, that's what I was getting at in the bit below the --- line in my
> patch. I think Aurelien was misled by the failure message and your email
> ([1] in my links above) which claimed that binutils would default to
> the 20191213 spec.
> It appears (and I'm not a tc person) that GCC must call GAS with the
> --misa-spec argument, and in GCC 12 the value used is 20191213.
> Either GCC 11 must pass --misa-spec=2.2 to binutils, or it passes
> nothing, leading binutils to be permissive about what -march=rv64i
> means.
I verified that "-misa-spec" is a new option introduced in GCC 12 and
the default value is set to 20191213 which is unfortunately backward
incompatible.
GCC 11 does not have the "-misa-spec" option, so I assume it produces
backward compatible codes.
IOW, what commit 6df2a016c0c8 was trying to fix has nothing to do with
binutils 2.38+. It's the GCC changes that is the culprit.
For this patch, I think it LGTM, so:
Reviewed-by: Bin Meng <bmeng@xxxxxxxxxxx>
>
> The permissive option would "seem" to be correct, as building with clang
> (that to my knowledge doesn't pass --misa-spec to GAS) and with
> -march=rv64i has no issues assembling.
>
> It'd appear to me that binutils is a *player* in this issue, but is not
> the culprit of the issue Aurelien sought to fix.
>
> I dunno what I am talking about though, this is all from playing around
> with many tc variants and see how it goes!
>
Regards,
Bin
