Hi Andrew,
Thanks for the feedback.
On Tue, 16 Jun 2020 at 09:33, Andrew Jones <drjones@xxxxxxxxxx> wrote:
>
> Hi Beata,
>
> I see Peter just picked this up, so I'm a bit late getting to it. I do
> have a couple comments below though.
>
> Thanks,
> drew
>
> On Fri, May 29, 2020 at 12:27:56PM +0100, Beata Michalska wrote:
> > On ARMv7 & ARMv8 some load/store instructions might trigger a data abort
> > exception with no valid ISS info to be decoded. The lack of decode info
> > makes it at least tricky to emulate those instruction which is one of the
> > (many) reasons why KVM will not even try to do so.
> >
> > Add support for handling those by requesting KVM to inject external
> > dabt into the quest.
> >
> > Signed-off-by: Beata Michalska <beata.michalska@xxxxxxxxxx>
> > ---
> > target/arm/cpu.h | 2 ++
> > target/arm/kvm.c | 64 +++++++++++++++++++++++++++++++++++++++++++++++++++-
> > target/arm/kvm_arm.h | 11 +++++++++
> > 3 files changed, 76 insertions(+), 1 deletion(-)
> >
> > diff --git a/target/arm/cpu.h b/target/arm/cpu.h
> > index 677584e..3702f21 100644
> > --- a/target/arm/cpu.h
> > +++ b/target/arm/cpu.h
> > @@ -570,6 +570,8 @@ typedef struct CPUARMState {
> > uint64_t esr;
> > } serror;
> >
> > + uint8_t ext_dabt_pending; /* Request for injecting ext DABT */
> > +
> > /* State of our input IRQ/FIQ/VIRQ/VFIQ lines */
> > uint32_t irq_line_state;
> >
> > diff --git a/target/arm/kvm.c b/target/arm/kvm.c
> > index 4bdbe6d..bf84224 100644
> > --- a/target/arm/kvm.c
> > +++ b/target/arm/kvm.c
> > @@ -39,6 +39,7 @@ const KVMCapabilityInfo kvm_arch_required_capabilities[] = {
> >
> > static bool cap_has_mp_state;
> > static bool cap_has_inject_serror_esr;
> > +static bool cap_has_inject_ext_dabt;
> >
> > static ARMHostCPUFeatures arm_host_cpu_features;
> >
> > @@ -244,6 +245,16 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
> > ret = -EINVAL;
> > }
> >
> > + if (kvm_check_extension(s, KVM_CAP_ARM_NISV_TO_USER)) {
> > + if (kvm_vm_enable_cap(s, KVM_CAP_ARM_NISV_TO_USER, 0)) {
> > + error_report("Failed to enable KVM_CAP_ARM_NISV_TO_USER cap");
> > + } else {
> > + /* Set status for supporting the external dabt injection */
> > + cap_has_inject_ext_dabt = kvm_check_extension(s,
> > + KVM_CAP_ARM_INJECT_EXT_DABT);
> > + }
> > + }
> > +
> > return ret;
> > }
> >
> > @@ -703,9 +714,16 @@ int kvm_put_vcpu_events(ARMCPU *cpu)
> > events.exception.serror_esr = env->serror.esr;
> > }
> >
> > + if (cap_has_inject_ext_dabt) {
> > + events.exception.ext_dabt_pending = env->ext_dabt_pending;
> > + }
> > +
> > ret = kvm_vcpu_ioctl(CPU(cpu), KVM_SET_VCPU_EVENTS, &events);
> > if (ret) {
> > error_report("failed to put vcpu events");
> > + } else {
> > + /* Clear instantly if the call was successful */
> > + env->ext_dabt_pending = 0;
> > }
> >
> > return ret;
> > @@ -819,7 +837,12 @@ int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run)
> > ret = EXCP_DEBUG;
> > } /* otherwise return to guest */
> > break;
> > - default:
> > + case KVM_EXIT_ARM_NISV:
> > + /* External DABT with no valid iss to decode */
> > + ret = kvm_arm_handle_dabt_nisv(cs, run->arm_nisv.esr_iss,
> > + run->arm_nisv.fault_ipa);
> > + break;
> > + default:
> > qemu_log_mask(LOG_UNIMP, "%s: un-handled exit reason %d\n",
> > __func__, run->exit_reason);
> > break;
> > @@ -955,3 +978,42 @@ int kvm_arch_msi_data_to_gsi(uint32_t data)
> > {
> > return (data - 32) & 0xffff;
> > }
> > +
> > +int kvm_arm_handle_dabt_nisv(CPUState *cs, uint64_t esr_iss,
> > + uint64_t fault_ipa)
>
> This function could be static since it's in the same file as its one
> and only caller.
>
Right. Will do.
> > +{
> > + ARMCPU *cpu = ARM_CPU(cs);
> > + CPUARMState *env = &cpu->env;
> > +
> > + /*
> > + * ISS [23:14] is invalid so there is a limited info
> > + * on what has just happened so the only *useful* thing that can
> > + * be retrieved from ISS is WnR & DFSC (though in some cases WnR
> > + * might be less of a value as well)
> > + */
> > +
> > + /*
> > + * Request KVM to inject the external data abort into the guest
> > + * by setting a pending exception on the affected vcpu.
> > + */
> > + if (cap_has_inject_ext_dabt) {
> > + /* Set pending exception */
> > + env->ext_dabt_pending = 1;
> > + /*
> > + * Even though at this point, the vcpu regs are out of sync,
> > + * directly calling the KVM_SET_VCPU_EVENTS ioctl without
> > + * explicitly synchronizing those, is enough and it also avoids
> > + * overwriting changes done by KVM.
> > + * The vcpu is not being marked as 'dirty' as all the changes
> > + * needed to inject the abort are being handled by KVM only
> > + * and there is no need for syncing either way
> > + */
> > + return kvm_put_vcpu_events(cpu);
>
> This looks fragile. What if we change kvm_put_vcpu_events() in a way
> that requires CPU synchronization? Or, what if we change something else
> that results in the VCPU getting flagged as dirty before finishing this
> run loop, and then we end up calling kvm_put_vcpu_events() twice, possibly
> causing some other problem?
>
> I think I'd prefer seeing the KVM_SET_VCPU_EVENTS ioctl get called
> directly from here with the events set to only inject the dabt, as we
> know that will be handled immediately and doesn't require any CPU
> synchronization.
>
It might be 'fragile' though I would expect any changes to kvm_put_vcpu_events
revisit all users, but I have no objections to calling ioctl directly
here if that's
the prefered way.
Will try to push the changes sometime later this week.
Thanks again for the review.
BR
Beata
> Thanks,
> drew
>
> > + } else {
> > + error_report("Data abort exception triggered by guest memory access "
> > + "at physical address: 0x" TARGET_FMT_lx,
> > + (target_ulong)fault_ipa);
> > + error_printf("KVM unable to emulate faulting instruction.\n");
> > + return -1;
> > + }
> > +}
> > diff --git a/target/arm/kvm_arm.h b/target/arm/kvm_arm.h
> > index 48bf5e1..e939e51 100644
> > --- a/target/arm/kvm_arm.h
> > +++ b/target/arm/kvm_arm.h
> > @@ -453,6 +453,17 @@ struct kvm_guest_debug_arch;
> > void kvm_arm_copy_hw_debug_data(struct kvm_guest_debug_arch *ptr);
> >
> > /**
> > + * kvm_arm_handle_dabt_nisv:
> > + * @cs: CPUState
> > + * @esr_iss: ISS encoding (limited) for the exception from Data Abort
> > + * ISV bit set to '0b0' -> no valid instruction syndrome
> > + * @fault_ipa: faulting address for the synch data abort
> > + *
> > + * Returns: 0 if the exception has been handled, < 0 otherwise
> > + */
> > +int kvm_arm_handle_dabt_nisv(CPUState *cs, uint64_t esr_iss,
> > + uint64_t fault_ipa);
> > +/**
> > * its_class_name:
> > *
> > * Return the ITS class name to use depending on whether KVM acceleration
> > --
> > 2.7.4
> >
> >
>
_______________________________________________
kvmarm mailing list
kvmarm@xxxxxxxxxxxxxxxxxxxxx
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
