hi kevin,
i set a breakpint at bmdma_active_if. the first 2 breaks encountered
when the last path in the multipath
failed, but the assertion was not true.
when i kicked one path back in the breakpoint was reached again, this
time leading to an assert.
the stacktrace is from the point shortly before.
hope this helps.
br,
peter
--
(gdb) b bmdma_active_if
Breakpoint 2 at 0x43f2e0: file
/usr/src/qemu-kvm-0.12.3/hw/ide/internal.h, line 507.
(gdb) c
Continuing.
[Switching to Thread 0x7f7b3300d950 (LWP 21171)]
Breakpoint 2, bmdma_active_if (bmdma=0xe31fd8) at
/usr/src/qemu-kvm-0.12.3/hw/ide/internal.h:507
507 assert(bmdma->unit != (uint8_t)-1);
(gdb) c
Continuing.
Breakpoint 2, bmdma_active_if (bmdma=0xe31fd8) at
/usr/src/qemu-kvm-0.12.3/hw/ide/internal.h:507
507 assert(bmdma->unit != (uint8_t)-1);
(gdb) c
Continuing.
Breakpoint 2, bmdma_active_if (bmdma=0xe31fd8) at
/usr/src/qemu-kvm-0.12.3/hw/ide/internal.h:507
507 assert(bmdma->unit != (uint8_t)-1);
(gdb) bt full
#0 bmdma_active_if (bmdma=0xe31fd8) at
/usr/src/qemu-kvm-0.12.3/hw/ide/internal.h:507
__PRETTY_FUNCTION__ = "bmdma_active_if"
#1 0x000000000043f6ba in ide_read_dma_cb (opaque=0xe31fd8, ret=0) at
/usr/src/qemu-kvm-0.12.3/hw/ide/core.c:554
bm = (BMDMAState *) 0xe31fd8
s = (IDEState *) 0xe17940
n = 0
sector_num = 0
#2 0x000000000058730c in dma_bdrv_cb (opaque=0xe17940, ret=0) at
/usr/src/qemu-kvm-0.12.3/dma-helpers.c:94
dbs = (DMAAIOCB *) 0xe17940
cur_addr = 0
cur_len = 0
mem = (void *) 0x0
#3 0x000000000049e510 in qemu_laio_process_completion (s=0xe119c0,
laiocb=0xe179c0) at linux-aio.c:68
ret = 0
#4 0x000000000049e611 in qemu_laio_enqueue_completed (s=0xe119c0,
laiocb=0xe179c0) at linux-aio.c:107
No locals.
#5 0x000000000049e787 in qemu_laio_completion_cb (opaque=0xe119c0) at
linux-aio.c:144
iocb = (struct iocb *) 0xe179f0
laiocb = (struct qemu_laiocb *) 0xe179c0
val = 1
ret = 8
nevents = 1
i = 0
events = {{data = 0x0, obj = 0xe179f0, res = 4096, res2 = 0}, {data
= 0x0, obj = 0x0, res = 0, res2 = 0} <repeats 46 times>, {data = 0x0,
obj = 0x0, res = 0,
res2 = 4365191}, {data = 0x429abf, obj = 0x7f7b3300c410, res =
4614129721674825936, res2 = 14777248}, {data = 0x3000000018, obj =
0x7f7b3300c4c0, res = 140167113393152,
res2 = 47259417504}, {data = 0xe17740, obj = 0xa3300c4e0, res =
140167113393184, res2 = 0}, {data = 0xe17740, obj = 0x0, res = 0, res2 =
17}, {data = 0x7f7b3300ccf0,
obj = 0x92, res = 32, res2 = 168}, {data = 0x7f7b33797a00, obj =
0x801000, res = 0, res2 = 140167141433408}, {data = 0x7f7b34496e00, obj
= 0x7f7b33797a00,
res = 140167113393392, res2 = 8392704}, {data = 0x0, obj =
0x7f7b34aca040, res = 140167134932480, res2 = 140167118209654}, {data =
0x7f7b3300d950, obj = 0x42603d, res = 0,
res2 = 42949672960}, {data = 0x7f7b3300c510, obj = 0xe17ba0, res =
14776128, res2 = 43805361568}, {data = 0x7f7b3300ced0, obj = 0x42797e,
res = 0, res2 = 14777248}, {
data = 0x174, obj = 0x0, res = 373, res2 = 0}, {data = 0x176, obj =
0x0, res = 3221225601, res2 = 0}, {data = 0x4008ae89c0000083, obj = 0x0,
res = 209379655938, res2 = 0}, {
data = 0x7f7bc0000084, obj = 0x0, res = 3221225602, res2 = 0}, {data
= 0x7f7b00000012, obj = 0x0, res = 17, res2 = 0}, {data = 0x0, obj =
0x11, res = 140167113395840,
res2 = 146}, {data = 0x20, obj = 0xa8, res = 140167121304064, res2 =
8392704}, {data = 0x0, obj = 0x7f7b34aca040, res = 140167134932480, res2
= 140167121304064}, {
data = 0x7f7b3300c680, obj = 0x801000, res = 0, res2 =
140167141433408}, {data = 0x7f7b34496e00, obj = 0x7f7b334a4276, res =
140167113398608, res2 = 4350013}, {data = 0x0,
obj = 0xa00000000, res = 140167113393824, res2 = 14777248}, {data =
0xe2c010, obj = 0xa3300c730, res = 140167113396320, res2 = 4356478},
{data = 0x0, obj = 0xe17ba0,
res = 372, res2 = 0}, {data = 0x175, obj = 0x0, res = 374, res2 =
0}, {data = 0xc0000081, obj = 0x0, res = 3221225603, res2 = 0}, {data =
0xc0000102, obj = 0x0,
res = 3221225604, res2 = 0}, {data = 0xc0000082, obj = 0x0, res =
18, res2 = 0}, {data = 0x11, obj = 0x0, res = 0, res2 = 0}, {data = 0x0,
obj = 0x0, res = 0, res2 = 0}, {
data = 0x0, obj = 0x0, res = 0, res2 = 0}, {data = 0x0, obj = 0x0,
res = 0, res2 = 0}, {data = 0x0, obj = 0x0, res = 0, res2 = 0}, {data =
0x0, obj = 0x0, res = 0,
res2 = 140167139245116}, {data = 0x0, obj = 0x7f7b34abe118, res = 9,
res2 = 13}, {data = 0x25bf5fc6, obj = 0x7f7b348b40f0, res =
140167117719264, res2 = 6}, {
data = 0x96fd7f, obj = 0x7f7b3300c850, res = 140167113394680, res2 =
140167117724520}, {data = 0x0, obj = 0x7f7b34abe168, res =
140167141388288, res2 = 4206037}, {
data = 0x7f7b3343a210, obj = 0x402058, res = 21474836480, res2 =
4294968102}, {data = 0x0, obj = 0x7f7b34ac8358, res = 140167113394736,
res2 = 140167113394680}, {
data = 0x25bf5fc6, obj = 0x7f7b3300c9e0, res = 0, res2 =
140167139246910}, {data = 0x0, obj = 0x7f7b34abe168, res = 5, res2 =
140167139245116}, {data = 0x1,
obj = 0x7f7b34abe120, res = 10, res2 = 13}, {data = 0x9fd7b9dd, obj
= 0x7f7b348b40f0, res = 140167139205904, res2 = 140166257704989}, {data
= 0x27f5ee7,
obj = 0x7f7b3300c950, res = 140167113394936, res2 =
140167139205920}, {data = 0x0, obj = 0x7f7b34abe510, res =
140167141434664, res2 = 140167134874710}, {
data = 0x7f7b348aa5a8, obj = 0x7f7b34486e30, res = 21474836480, res2
= 4294967319}, {data = 0x7f7b3428427c, obj = 0x7f7b34ac7cc8, res =
140167113394992,
res2 = 140167113394936}, {data = 0x9fd7b9dd, obj = 0x7f7b3300cae0,
res = 0, res2 = 140167139246910}, {data = 0x0, obj = 0x7f7b34abe510, res
= 140166257704965, res2 = 0}, {
data = 0x500000001, obj = 0xffffffff, res = 0, res2 = 8627912},
{data = 0x801000, obj = 0x100000000, res = 140167141385488, res2 =
140167141424328}, {data = 0x7f7b3300cb60,
obj = 0x7f7b34ac7970, res = 140167134874710, res2 = 0}, {data = 0x5,
obj = 0x0, res = 140167117743864, res2 = 140167113398608}, {data =
0x7f7b3300cb00, obj = 0x7f7b348bf592,
res = 14665264, res2 = 90}, {data = 0x2, obj = 0x7f7b33508255, res =
140167113398608, res2 = 140167113394944}, {data = 0x801000, obj = 0x0,
res = 140167141433408,
res2 = 4966659}, {data = 0x50, obj = 0x23300cb2f, res =
140167139206472, res2 = 140167141434664}, {data = 0x98, obj =
0xffffffff, res = 140167113395072, res2 = 2191368}, {
data = 0x801000, obj = 0x3, res = 140167141433408, res2 =
140167139245116}, {data = 0x7f7b34486000, obj = 0x7f7b34abe0e8, res = 3,
res2 = 13}, {data = 0xa896c0a2,
obj = 0x7f7b348b40f0, res = 140167132758616, res2 = 34}, {data =
0x2a25b02, obj = 0x7f7b3300cb90, res = 140167113395512, res2 =
140167132758672}, {data = 0x0,
obj = 0x7f7b34abe1b0, res = 140167141396480, res2 = 4204852}, {data
= 0x7f7b34284458, obj = 0x400d38, res = 21474836480, res2 = 4294967302},
{data = 0xc008ae67325339e0,
---Type <return> to continue, or q <return> to quit---
obj = 0x7f7b34ac8358, res = 140167113395568, res2 =
140167113395512}, {data = 0xa896c0a2, obj = 0x7f7b3300cd20, res = 0,
res2 = 140167139246910}, {data = 0x0,
obj = 0x7f7b34abe1b0, res = 140166257704965, res2 = 0}, {data = 0x1,
obj = 0x101010101010101, res = 140167113395504, res2 = 14776768}, {data
= 0x7f7b3300cd20,
obj = 0x13300cd7c, res = 140167141384624, res2 = 140167141426008},
{data = 0x7f7b3300cda0, obj = 0x7f7b34ac8000, res = 4204852, res2 =
4364819}, {data = 0x42994b,
obj = 0x7f7b3300ccf0, res = 13837501612500713496, res2 = 14658720},
{data = 0x3000000018, obj = 0x7f7b3300cda0, res = 140167113395424, res2
= 4371109}, {
data = 0x7f7b3300cdc0, obj = 0x3300cd4c, res = 64424509441, res2 =
4364819}, {data = 0x42994b, obj = 0x7f7b3300cd50, res =
13837501612500713473, res2 = 14658720}, {
data = 0x3000000018, obj = 0x7f7b3300ce00, res = 140167132758816,
res2 = 140167141396480}, {data = 0x7f7b3300ce20, obj = 0xffffffff, res =
140167113395580, res2 = 8626296}, {
data = 0x801000, obj = 0x0, res = 140167141433408, res2 =
140167134932480}, {data = 0x0, obj = 0x7f7b348b871a, res =
140166257704965, res2 = 0}, {data = 0x7f7b3300cdd0,
obj = 0x42b2a5, res = 140167132758816, res2 = 140167113398608}}
ts = {tv_sec = 0, tv_nsec = 0}
s = (struct qemu_laio_state *) 0xe119c0
#6 0x000000000049e841 in laio_cancel (blockacb=0xe179c0) at linux-aio.c:184
laiocb = (struct qemu_laiocb *) 0xe179c0
event = {data = 0x1, obj = 0x4c7fb1, res = 140167113395792, res2 =
4384262}
ret = -22
#7 0x000000000049a29b in bdrv_aio_cancel (acb=0xe179c0) at block.c:1792
No locals.
#8 0x000000000058755a in dma_aio_cancel (acb=0xe17940) at
/usr/src/qemu-kvm-0.12.3/dma-helpers.c:138
dbs = (DMAAIOCB *) 0xe17940
#9 0x000000000049a29b in bdrv_aio_cancel (acb=0xe17940) at block.c:1792
No locals.
#10 0x0000000000444a0c in ide_dma_cancel (bm=0xe31fd8) at
/usr/src/qemu-kvm-0.12.3/hw/ide/core.c:2838
No locals.
#11 0x0000000000444f39 in bmdma_cmd_writeb (opaque=0xe31fd8, addr=49152,
val=8) at /usr/src/qemu-kvm-0.12.3/hw/ide/pci.c:44
bm = (BMDMAState *) 0xe31fd8
#12 0x00000000004c81bc in ioport_write (index=0, address=49152, data=8)
at ioport.c:80
func = (IOPortWriteFunc *) 0x444f0c <bmdma_cmd_writeb>
default_func = {0x4c81d0 <default_ioport_writeb>, 0x4c8225
<default_ioport_writew>, 0x4c8282 <default_ioport_writel>}
#13 0x00000000004c8543 in cpu_outb (addr=49152, val=8 '\b') at ioport.c:198
No locals.
#14 0x0000000000429689 in kvm_handle_io (port=49152,
data=0x7f7b34ab7000, direction=1, size=1, count=1) at
/usr/src/qemu-kvm-0.12.3/kvm-all.c:535
i = 0
ptr = (uint8_t *) 0x7f7b34ab7000 "\b"
#15 0x000000000042bac3 in kvm_run (env=0xe17ba0) at
/usr/src/qemu-kvm-0.12.3/qemu-kvm.c:964
r = 0
kvm = (kvm_context_t) 0xdfb0d0
run = (struct kvm_run *) 0x7f7b34ab6000
fd = 15
#16 0x000000000042cdda in kvm_cpu_exec (env=0xe17ba0) at
/usr/src/qemu-kvm-0.12.3/qemu-kvm.c:1647
r = 0
#17 0x000000000042d564 in kvm_main_loop_cpu (env=0xe17ba0) at
/usr/src/qemu-kvm-0.12.3/qemu-kvm.c:1889
run_cpu = 1
#18 0x000000000042d6a5 in ap_main_loop (_env=0xe17ba0) at
/usr/src/qemu-kvm-0.12.3/qemu-kvm.c:1939
env = (struct CPUX86State *) 0xe17ba0
signals = {__val = {18446744067267100671, 18446744073709551615
<repeats 15 times>}}
data = (struct ioperm_data *) 0x0
#19 0x00007f7b3448d3ba in start_thread () from /lib/libpthread.so.0
No symbol table info available.
#20 0x00007f7b3350ffcd in clone () from /lib/libc.so.6
No symbol table info available.
#21 0x0000000000000000 in ?? ()
No symbol table info available.
(gdb) c
Continuing.
kvm: /usr/src/qemu-kvm-0.12.3/hw/ide/internal.h:507: bmdma_active_if:
Assertion `bmdma->unit != (uint8_t)-1' failed.
Program received signal SIGABRT, Aborted.
0x00007f7b3345cfb5 in raise () from /lib/libc.so.6
Kevin Wolf wrote:
Am 03.05.2010 23:26, schrieb Peter Lieven:
Hi Qemu/KVM Devel Team,
i'm using qemu-kvm 0.12.3 with latest Kernel 2.6.33.3.
As backend we use open-iSCSI with dm-multipath.
Multipath is configured to queue i/o if no path is available.
If we create a failure on all paths, qemu starts to consume 100%
CPU due to i/o waits which is ok so far.
1 odd thing: The Monitor Interface is not responding any more ...
What es a really blocker is that KVM crashes with:
kvm: /usr/src/qemu-kvm-0.12.3/hw/ide/internal.h:507: bmdma_active_if:
Assertion `bmdma->unit != (uint8_t)-1' failed.
after the multipath has reestablisched at least one path.
Can you get a stack backtrace with gdb?
Any ideas? I remember this was working with earlier kernel/kvm/qemu
versions.
If it works in the same setup with an older qemu version, bisecting
might help.
Kevin
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
