Hi,
On Mon, Apr 11, 2022 at 10:12:13AM +0100, Will Deacon wrote:
> Hi Sean,
>
> Cheers for the heads-up.
>
> [+Marc and Alex as this looks similar to [1]]
>
> On Fri, Apr 08, 2022 at 05:01:21PM +0000, Sean Christopherson wrote:
> > On Fri, Apr 08, 2022, Peter Gonda wrote:
> > > On Thu, Apr 7, 2022 at 8:55 PM Sean Christopherson <seanjc@xxxxxxxxxx> wrote:
> > > > On Thu, Apr 07, 2022, Peter Gonda wrote:
> > > > > If an SEV-ES guest requests termination, exit to userspace with
> > > > > KVM_EXIT_SYSTEM_EVENT and a dedicated SEV_TERM type instead of -EINVAL
> > > > > so that userspace can take appropriate action.
> > > > >
> > > > > See AMD's GHCB spec section '4.1.13 Termination Request' for more details.
> > > >
> > > > Maybe it'll be obvious by the lack of compilation errors, but the changelog should
> > > > call out the flags => ndata+data shenanigans, otherwise this looks like ABI breakage.
> > >
> > > Hmm I am not sure we can do this change anymore given that we have two
> > > call sites using 'flags'
> > >
> > > arch/arm64/kvm/psci.c:184
> > > arch/riscv/kvm/vcpu_sbi.c:97
> > >
> > > I am not at all familiar with ARM and RISC-V but some quick reading
> > > tells me these archs also require 64-bit alignment on their 64-bit
> > > accesses. If thats correct, should I fix this call sites up by
> > > proceeding with this ndata + data[] change and move whatever they are
> > > assigning to flags into data[0] like I am doing here? It looks like
> > > both of these changes are not in a kernel release so IIUC we can still
> > > fix the ABI here?
> >
> > Yeah, both came in for v5.18. Given that there will be multiple paths that need
> > to set data, it's worth adding a common helper to the dirty work.
> >
> > Anup and Will,
> >
> > system_event.flags is broken (at least on x86) due to the prior 'type' field not
> > being propery padded, e.g. userspace will read/write garbage if the userspace
> > and kernel compilers pad structs differently.
> >
> > struct {
> > __u32 type;
> > __u64 flags;
> > } system_event;
>
> On arm64, I think the compiler is required to put the padding between type
> and flags so that both the struct and 'flags' are 64-bit aligned [2]. Does
> x86 not offer any guarantees on the overall structure alignment?
This is also my understanding. The "Procedure Call Standard for the Arm
64-bit Architecture" [1] has these rules for structs (called "aggregates"):
"An aggregate, where the members are laid out sequentially in memory
(possibly with inter-member padding)" => the field "type" is at offset 0 in
the struct.
"The member alignment of an element of a composite type is the alignment of
that member after the application of any language alignment modifiers to
that member" => "flags" is 8-byte aligned and "type" is 4-byte aligned.
"The alignment of an aggregate shall be the alignment of its most-aligned
member." => struct system_event is 8-byte aligned.
and finally:
"The size of an aggregate shall be the smallest multiple of its alignment
that is sufficient to hold all of its members."
I think this is the only possible layout that satisfies all the above
conditions:
offset 0-3: type (at an 8-byte aligned address in memory)
offset 4-7: padding
offset 8-15: flags
so under all compilers that correctly implement the architecture the struct
will have the same layout.
[1] https://github.com/ARM-software/abi-aa/releases/download/2021Q3/aapcs64.pdf
>
> > Our plan to unhose this is to change the struct as follows and use bit 31 in the
> > 'type' to indicate that ndata+data are valid.
> >
> > struct {
> > __u32 type;
> > __u32 ndata;
> > __u64 data[16];
> > } system_event;
> >
> > Any objection to updating your architectures to use a helper to set the bit and
> > populate ndata+data accordingly? It'll require a userspace update, but v5.18
> > hasn't officially released yet so it's not kinda sort not ABI breakage.
>
> It's a bit annoying, as we're using the current structure in Android 13 :/
> Obviously, if there's no choice then upstream shouldn't worry, but it means
> we'll have to carry a delta in crosvm. Specifically, the new 'ndata' field
> is going to be unusable for us because it coincides with the padding.
Just a thought, but wouldn't such a drastical change be better implemented
as a new exit_reason and a new associated struct?
Thanks,
Alex
>
> Will
>
> [1] https://lore.kernel.org/r/20220407162327.396183-6-alexandru.elisei@xxxxxxx
> [2] https://github.com/ARM-software/abi-aa/blob/60a8eb8c55e999d74dac5e368fc9d7e36e38dda4/aapcs64/aapcs64.rst#composite-types
