Re: [vfio-users] Interrupt Handling: Regression or config change?

Ede Wolf <listac@xxxxxxxxxxxxxxxx> · Sun, 15 Jan 2017 00:03:54 +0100

Hello Alex,

thanks for the fast reply (and my sympathie, if you've written this from 
work on a saturday).

Anyway, adding vector_hashing=0 to the kvm module did the trick! 
Passthrough is working properly again with a recent kernel.

# uname -a
Linux gewitterhexe 4.9.3-1-ARCH #1 SMP PREEMPT Thu Jan 12 21:34:12 CET 
2017 x86_64 GNU/Linux

Without that module option this very kernel does not work (as none so 
far after 4.5.7). Also tested and verified, of course

I'll leave that option activated now until further notice.

Thanks to all!

Ede

Am 14.01.2017 um 23:18 schrieb Alex Williamson:
Hi Feng Wu,

Please see the device assignment interrupt regression described in this
thread:

https://www.redhat.com/archives/vfio-users/2017-January/msg00004.html

As seen below, Ede has bisected back to commit 520040146a0a as
introducing this regression.

Ede, thanks for performing the bisect.  For further validation, this
commit introduced the "vector_hashing" module option to KVM which
appears to disable the new behavior.  Can you try loading the kvm
module with vector_hashing=0, ex.

modprobe.d/local.conf:
options kvm vector_hashing=0

(followed by reboot) to verify?  Thanks,

Alex

On Sat, 14 Jan 2017 21:52:45 +0100
Ede Wolf <listac@xxxxxxxxxxxxxxxx> wrote:

Hello,

it seems, we have a result! Hooray. Unless I've done something wrong,
which is not unlikely at all. Which ever way, what will be next?

But before the late braking news, thanks to Laszlo for further detailed
information and of course for the heads up on the spelling. After
writing "dissecting" correctly, google was much more friendly.
Surprisingly... a bit embarassing that I cannot read properly, but once
you've made up your mind, one tends to read what he thinks has been
written.

Other issues: git fetch did not work for me, as it downloads a binary
repository, that of course cannot be configured or compiled.
So I choose git clone, but have not been able to limit the tags. If
somebody knows how to do that, I am all ear.
Also the required Mergebase took some time to get along with. Not sure
that I've dealt correctly with it (at the beginning of the full log, in
case someone wants to verify).

Back on topic, the last two entries, that by description fit just
perfectly. Very sorry for having forgotten to set the language to "C"
beforehand. I hope it's still clear. Otherwise I'll translate.
The upper cases are only for this mail.

#[root@archbuild linux-stable]# git describe --tag
#v4.5-rc3-12-g23a1c2579b57
#[root@archbuild linux-stable]# git bisect GOOD v4.5-rc3-12-g23a1c2579b57
#binäre Suche: danach noch 0 Commits zum Testen übrig (ungefähr 1 Schritt)
#[6228a0da805792c2f25b32e9b926d0810a6648ab] KVM: x86: Add
lowest-priority support for vt-d posted-interrupts
#[root@archbuild linux-stable]#
#============================
#[root@archbuild linux-stable]# git describe --tag
#v4.5-rc3-14-g6228a0da8057
#[root@archbuild linux-stable]# git bisect BAD v4.5-rc3-14-g6228a0da8057
#binäre Suche: danach noch 0 Commits zum Testen übrig (ungefähr 0 Schritte)
#[520040146a0af36f7875ec06b58f44b19a0edf53] KVM: x86: Use vector-hashing
to deliver lowest-priority interrupts
#[root@archbuild linux-stable]#

In short, while Vector Linux is quite cool, vector hashing seems not to
be. Maybe Mr. Wu did not imagine someone still is using an AMD CPU, as I
do.

# -----------------

[root@archbuild linux-stable]# git describe --tag
v4.5-rc3-13-g520040146a0a
[root@archbuild linux-stable]# git bisect bad v4.5-rc3-13-g520040146a0a
520040146a0af36f7875ec06b58f44b19a0edf53 is the first bad commit
commit 520040146a0af36f7875ec06b58f44b19a0edf53
Author: Feng Wu <feng.wu@xxxxxxxxx>
Date:   Mon Jan 25 16:53:33 2016 +0800

     KVM: x86: Use vector-hashing to deliver lowest-priority interrupts

     Use vector-hashing to deliver lowest-priority interrupts, As an
     example, modern Intel CPUs in server platform use this method to
     handle lowest-priority interrupts.

     Signed-off-by: Feng Wu <feng.wu@xxxxxxxxx>
     Reviewed-by: Radim Krčmář <rkrcmar@xxxxxxxxxx>
     Signed-off-by: Paolo Bonzini <pbonzini@xxxxxxxxxx>

:040000 040000 f1d833ca902832009798f3aacdf1ddd4159e4202
3130cfbde26ed6c4f0b2fae55c9670223e0ff87d M      arch

# -----------------

And finally the somewhat (some early, all bad sects are missing)
complete log. Again thanks very much for all that helped so far:

#[root@archbuild src]# git init
#[root@archbuild src]# git clone
git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
#[root@archbuild src]# cd linux-stable/
#[root@archbuild linux-stable]# git bisect start
#[root@archbuild linux-stable]# git bisect good v4.5.7
#[root@archbuild linux-stable]# git bisect bad v4.6
#binäre Suche: eine Merge-Basis muss geprüft werden
#[b562e44f507e863c6792946e4e1b1449fbbac85d] Linux 4.5
#[root@archbuild linux-stable]# git bisect good
b562e44f507e863c6792946e4e1b1449fbbac85d
#binäre Suche: danach noch 8131 Commits zum Testen übrig (ungefähr 13
Schritte)
#[6b5f04b6cf8ebab9a65d9c0026c650bb2538fd0f] Merge branch 'for-4.6' of
git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup
#[root@archbuild linux-stable]# git bisect good v4.5.7
#binäre Suche: danach noch 8131 Commits zum Testen übrig (ungefähr 13
Schritte)
#[6b5f04b6cf8ebab9a65d9c0026c650bb2538fd0f] Merge branch 'for-4.6' of
git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup
#[root@archbuild linux-stable]#

# ...

#============================
#[root@archbuild linux-stable]# git bisect bad v4.5-3326-g96b9b1c95660
#binäre Suche: danach noch 1605 Commits zum Testen übrig (ungefähr 11
Schritte)
#[277edbabf6fece057b14fb6db5e3a34e00f42f42] Merge tag
'pm+acpi-4.6-rc1-1' of
git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm
#============================
#linux-stable]# git describe --tag
#v4.5-1720-g277edbabf6fe
#[root@archbuild linux-stable]# git bisect bad v4.5-1720-g277edbabf6fe
#binäre Suche: danach noch 879 Commits zum Testen übrig (ungefähr 10
Schritte)
#[5ca5446ec5ba5e79a6f271cd026bb153d6850fcc] Merge tag 'pinctrl-v4.6-1'
of git://git.kernel.org/pub/scm/linux/kernel/git/linusw/linux-pinctrl
#[root@archbuild linux-stable]#
#============================
#[root@archbuild linux-stable]# git describe --tag
#v4.5-840-g5ca5446ec5ba
#[root@archbuild linux-stable]# git bisect good v4.5-840-g5ca5446ec5ba
#binäre Suche: danach noch 356 Commits zum Testen übrig (ungefähr 9
Schritte)
#[10dc3747661bea9215417b659449bb7b8ed3df2c] Merge tag 'for-linus' of
git://git.kernel.org/pub/scm/virt/kvm/kvm
#[root@archbuild linux-stable]#
#============================
#[root@archbuild linux-stable]# git describe --tag
#v4.5-1363-g10dc3747661b
#[root@archbuild linux-stable]# git bisect bad v4.5-1363-g10dc3747661b
#binäre Suche: danach noch 255 Commits zum Testen übrig (ungefähr 8
Schritte)
#[13f6f62f61b4d3d5f45bed889128bb7ff3fda5ed] Merge tag 'rtc-4.6' of
git://git.kernel.org/pub/scm/linux/kernel/git/abelloni/linux
#[root@archbuild linux-stable]#
#============================
#[root@archbuild linux-stable]# git describe --tag
#v4.5-1107-g13f6f62f61b4
#[root@archbuild linux-stable]# git bisect good v4.5-1107-g13f6f62f61b4
#binäre Suche: danach noch 142 Commits zum Testen übrig (ungefähr 7
Schritte)
#[bb9eadf0c35f2e7eb5ca6468f46ebb7473b85537] KVM: MMU: micro-optimize
gpte_access
#[root@archbuild linux-stable]#
#============================
#[root@archbuild linux-stable]# git describe --tag
#v4.5-rc3-123-gbb9eadf0c35f
#[root@archbuild linux-stable]# git bisect bad v4.5-rc3-123-gbb9eadf0c35f
#binäre Suche: danach noch 66 Commits zum Testen übrig (ungefähr 6 Schritte)
#[433da86023f866820e9bcd7f0889d944005d311c] KVM: async_pf: use
list_first_entry
#[root@archbuild linux-stable]#
#============================
#[root@archbuild linux-stable]# git describe --tag
#v4.5-rc3-56-g433da86023f8
#[root@archbuild linux-stable]# git bisect bad v4.5-rc3-56-g433da86023f8
#binäre Suche: danach noch 22 Commits zum Testen übrig (ungefähr 5 Schritte)
#[8a08b9c7379dc881ff5f00c086877353888a982f] KVM: s390: usage hint for
adapter mappings
#[root@archbuild linux-stable]#
#============================
#[root@archbuild linux-stable]# git describe --tag
#v4.5-rc3-33-g8a08b9c7379d
#[root@archbuild linux-stable]# git bisect bad v4.5-rc3-33-g8a08b9c7379d
#binäre Suche: danach noch 11 Commits zum Testen übrig (ungefähr 4 Schritte)
#[0e8bc06a2fbb4d6b688baa8e2416cd07f9453595] KVM: s390: PSW forwarding /
rewinding / ilc rework
#[root@archbuild linux-stable]#
#============================
#[root@archbuild linux-stable]# git describe --tag
#v4.5-rc3-21-g0e8bc06a2fbb
#[root@archbuild linux-stable]# git bisect bad v4.5-rc3-21-g0e8bc06a2fbb
#binäre Suche: danach noch 5 Commits zum Testen übrig (ungefähr 3 Schritte)
#[b6ce978067e75187d3c30f59b60d390a29374fab] KVM/VMX: Add host irq
information in trace event when updating IRTE for posted interrupts
#[root@archbuild linux-stable]#
#============================
#[root@archbuild linux-stable]# git describe --tag
#v4.5-rc3-15-gb6ce978067e7
#[root@archbuild linux-stable]# git bisect bad v4.5-rc3-15-gb6ce978067e7
#binäre Suche: danach noch 2 Commits zum Testen übrig (ungefähr 1 Schritt)
#[23a1c2579b575b228a6c685dfe93f296d3d5e0e1] KVM: Recover IRTE to
remapped mode if the interrupt is not single-destination
#============================
#[root@archbuild linux-stable]# git describe --tag
#v4.5-rc3-12-g23a1c2579b57
#[root@archbuild linux-stable]# git bisect good v4.5-rc3-12-g23a1c2579b57
#binäre Suche: danach noch 0 Commits zum Testen übrig (ungefähr 1 Schritt)
#[6228a0da805792c2f25b32e9b926d0810a6648ab] KVM: x86: Add
lowest-priority support for vt-d posted-interrupts
#[root@archbuild linux-stable]#
#============================
#[root@archbuild linux-stable]# git describe --tag
#v4.5-rc3-14-g6228a0da8057
#[root@archbuild linux-stable]# git bisect bad v4.5-rc3-14-g6228a0da8057
#binäre Suche: danach noch 0 Commits zum Testen übrig (ungefähr 0 Schritte)
#[520040146a0af36f7875ec06b58f44b19a0edf53] KVM: x86: Use vector-hashing
to deliver lowest-priority interrupts
#[root@archbuild linux-stable]#

Am 12.01.2017 um 19:49 schrieb Laszlo Ersek:
On 01/12/17 19:21, Ede Wolf wrote:
Besides my fighting with git, is it correct, that the working kernel is
labeld "bad", while the non working one "good"? May very well possible
for the workflow of dissect, just want to confirm, please

git bisect start
git bisect bad v4.5.4
git bisect good v4.6.1

No. git-bisect always locates a regression, that is, the first commit
that introduces a bug. Where things go from right to wrong. If this
matches your situation (old version works, new version breaks), simply
use the "good" and "bad" command line arguments verbatim.

Now, if you are looking for a commit that fixes a known bug, i.e., old
version is broken, new version works, but you don't exactly know what
fixed it, then you have to invert the meanings. Whenever the commit
under testing works, you have to say "bad", and whenever it breaks, you
have to say "good". More modern git versions help automate this (for
less boggling of the mind) with "git bisect terms". (See the manual.)
With older git versions, git command aliases, shell aliases, or small
wrapper scripts can hide the confusion.

Given that you are facing a real regression, you should use the good/bad
terms verbatim. From your earlier email, "4.5.7" works, "4.6" breaks,
hence you should qualify the former with "good", and the latter with "bad".

Also, the word is "bisect", not "dissect".

https://en.wiktionary.org/wiki/bisect
https://en.wiktionary.org/wiki/dissect

git-bisect performs a binary search, halving commit ranges. Hence the
"bi" in "bisect".

Laszlo

_______________________________________________
vfio-users mailing list
vfio-users@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/vfio-users

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html