On Wed, Feb 10, 2016 at 5:39 PM, Josh Cartwright <joshc@xxxxxx> wrote: > On Wed, Feb 10, 2016 at 09:33:28AM +0000, Jaggi, Manish wrote: >> I am trying to run a kvm guest on a host with 4.4 rt3 patchset >> applied. (Cavium thunderX arm64 system) Getting the following error: >> >> BUG: scheduling while atomic: qemu-system-aar/41889/0x00000002 >> [ 341.647463] Modules linked in: ipv6 thunderx_edac_lmc thunderx_edac_ccpi i2c_octeon edac_core shpchp aes_ce_blk ablk_helper cryptd aes_ce_cipher ghash_ce sha2_ce sha1_ce uio_pdrv_genirq rtc_efi uio >> [ 341.647477] Preemption disabled at:[<ffff8000000aba44>] kvm_vcpu_ioctl+0x30c/0x750 >> [ 341.647478] >> [ 341.647484] CPU: 2 PID: 41889 Comm: qemu-system-aar Not tainted 4.4.0-rt3-00120-gbb974fa #64 >> [ 341.647486] Hardware name: www.cavium.com ThunderX CRB1S/ThunderX CRB1S, BIOS 0.3 Dec 3 2015 >> [ 341.647488] Call trace: >> [ 341.647494] [<ffff800000097878>] dump_backtrace+0x0/0x160 >> [ 341.647499] [<ffff8000000979fc>] show_stack+0x24/0x30 >> [ 341.647503] [<ffff800000512608>] dump_stack+0x88/0xa8 >> [ 341.647509] [<ffff8000000f25c0>] __schedule_bug+0x70/0xc0 >> [ 341.647514] [<ffff8000008f8f38>] __schedule+0x510/0x580 >> [ 341.647517] [<ffff8000008f90e8>] schedule+0x50/0xf0 >> [ 341.647521] [<ffff8000008fa9a4>] rt_spin_lock_slowlock+0x124/0x2e0 >> [ 341.647525] [<ffff8000008fc5e0>] rt_spin_lock+0x60/0x70 >> [ 341.647530] [<ffff8000000bffe0>] kvm_vgic_flush_hwstate+0x60/0x278 >> [ 341.647535] [<ffff8000000b3140>] kvm_arch_vcpu_ioctl_run+0x108/0x618 >> [ 341.647547] [<ffff8000000aba44>] kvm_vcpu_ioctl+0x30c/0x750 >> [ 341.647553] [<ffff80000024b4dc>] do_vfs_ioctl+0x364/0x628 >> [ 341.647556] [<ffff80000024b834>] SyS_ioctl+0x94/0xa8 >> [ 341.647560] [<ffff800000093b04>] el0_svc_naked+0x38/0x3c >> >> The below patch enables preemption: >> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/arch/arm/kvm/arm.c?h=v4.4&id=1b3d546daf85ed2bc9966e12cee3e6435fb65eca > > Another relevant commit is 7e16aa81f9f6a7cfe2287b788a7d62abc2880185: > > Author: Christoffer Dall <christoffer.dall@xxxxxxxxxx> > > KVM: arm/arm64: Fix preemptible timer active state crazyness > > We were setting the physical active state on the GIC distributor in a > preemptible section, which could cause us to set the active state on > different physical CPU from the one we were actually going to run on, > hacoc ensues. > > Since we are no longer descheduling/scheduling soft timers in the > flush/sync timer functions, simply moving the timer flush into a > non-preemptible section. > > Reviewed-by: Marc Zyngier <marc.zyngier@xxxxxxx> > Signed-off-by: Christoffer Dall <christoffer.dall@xxxxxxxxxx> > >> arm/arm64: KVM: Properly account for guest CPU time >> >> Is there a way to do it without disabling preemption ? > > If the concern is touching the wrong per-CPU GIC distributor registers, > it should be sufficient on -rt to downgrade the preempt_disable() / > preempt_enable() to a migrate_disable() / migrate_enable(), which is > preemptible, but prevents the task from moving to another CPU. > Indeed! > > diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c > index 4f5c42a..2ce9cc2 100644 > --- a/arch/arm/kvm/arm.c > +++ b/arch/arm/kvm/arm.c > @@ -568,7 +568,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run) > * involves poking the GIC, which must be done in a > * non-preemptible context. > */ > - preempt_disable(); > + migrate_disable(); > kvm_timer_flush_hwstate(vcpu); > kvm_vgic_flush_hwstate(vcpu); > > @@ -587,7 +587,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run) > local_irq_enable(); > kvm_timer_sync_hwstate(vcpu); > kvm_vgic_sync_hwstate(vcpu); > - preempt_enable(); > + migrate_enable(); > continue; > } > > @@ -641,7 +641,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run) > > kvm_vgic_sync_hwstate(vcpu); > > - preempt_enable(); > + migrate_enable(); > > ret = handle_exit(vcpu, run, ret); > } > diff --git a/kernel/irq/manage.c b/kernel/irq/manage.c > index e4a0b8c..dec1156 100644 > --- a/kernel/irq/manage.c > +++ b/kernel/irq/manage.c > @@ -2135,7 +2135,7 @@ EXPORT_SYMBOL_GPL(irq_get_irqchip_state); > * This call sets the internal irqchip state of an interrupt, > * depending on the value of @which. > * > - * This function should be called with preemption disabled if the > + * This function should be called with migration disabled if the > * interrupt controller has per-cpu registers. > */ > int irq_set_irqchip_state(unsigned int irq, enum irqchip_irq_state which, Without knowing the details of RT or migrate_enable/disable, this looks fine to me. -Christoffer -- To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
