Thanks.
Sometimes when I observe multipage bvec on 5.3.7-301.fc31.x86_64.
This log is from Qemu virtio-blk.
========= size: 262144, iovcnt: 2
0: size: 229376 addr: 0x7fff6a7c8000
1: size: 32768 addr: 0x7fff64c00000
========= size: 262144, iovcnt: 2
0: size: 229376 addr: 0x7fff6a7c8000
1: size: 32768 addr: 0x7fff64c00000
I also tested on 5.6.0-0.rc6.git0.1.vanilla.knurd.1.fc31.x86_64.
And observe 64 iovcnt.
========= size: 262144, iovcnt: 64
0: size: 4096 addr: 0x7fffb5ece000
1: size: 4096 addr: 0x7fffb5ecd000
...
63: size: 4096 addr: 0x7fff8baec000
So I think this is a common issue of the upstream kernel, from 5.3 to 5.6.
BTW, I have used your script on 5.3.7-301.fc31.x86_64, it works well.
However, when updating to kernel 5.6.0-0.rc6.git0.1.vanilla.knurd.1.fc31.x86_64.
It complains:
root@192.168.19.239 16:57:23 ~ $ ./bvec_avg_pages.py
In file included from /virtual/main.c:2:
In file included from
/lib/modules/5.6.0-0.rc6.git0.1.vanilla.knurd.1.fc31.x86_64/build/include/uapi/linux/ptrace.h:142:
In file included from
/lib/modules/5.6.0-0.rc6.git0.1.vanilla.knurd.1.fc31.x86_64/build/arch/x86/include/asm/ptrace.h:5:
/lib/modules/5.6.0-0.rc6.git0.1.vanilla.knurd.1.fc31.x86_64/build/arch/x86/include/asm/segment.h:266:2:
error: expected '(' after 'asm'
alternative_io ("lsl %[seg],%[p]",
^
/lib/modules/5.6.0-0.rc6.git0.1.vanilla.knurd.1.fc31.x86_64/build/arch/x86/include/asm/alternative.h:240:2:
note: expanded from macro 'alternative_io'
asm_inline volatile (ALTERNATIVE(oldinstr, newinstr, feature) \
^
/lib/modules/5.6.0-0.rc6.git0.1.vanilla.knurd.1.fc31.x86_64/build/include/linux/compiler_types.h:210:24:
note: expanded from macro 'asm_inline'
#define asm_inline asm __inline
^
In file included from /virtual/main.c:3:
In file included from
/lib/modules/5.6.0-0.rc6.git0.1.vanilla.knurd.1.fc31.x86_64/build/include/linux/blkdev.h:5:
In file included from
/lib/modules/5.6.0-0.rc6.git0.1.vanilla.knurd.1.fc31.x86_64/build/include/linux/sched.h:14:
In file included from
/lib/modules/5.6.0-0.rc6.git0.1.vanilla.knurd.1.fc31.x86_64/build/include/linux/pid.h:5:
In file included from
/lib/modules/5.6.0-0.rc6.git0.1.vanilla.knurd.1.fc31.x86_64/build/include/linux/rculist.h:11:
In file included from
/lib/modules/5.6.0-0.rc6.git0.1.vanilla.knurd.1.fc31.x86_64/build/include/linux/rcupdate.h:27:
In file included from
/lib/modules/5.6.0-0.rc6.git0.1.vanilla.knurd.1.fc31.x86_64/build/include/linux/preempt.h:78:
In file included from
/lib/modules/5.6.0-0.rc6.git0.1.vanilla.knurd.1.fc31.x86_64/build/arch/x86/include/asm/preempt.h:7:
In file included from
/lib/modules/5.6.0-0.rc6.git0.1.vanilla.knurd.1.fc31.x86_64/build/include/linux/thread_info.h:38:
In file included from
/lib/modules/5.6.0-0.rc6.git0.1.vanilla.knurd.1.fc31.x86_64/build/arch/x86/include/asm/thread_info.h:12:
In file included from
/lib/modules/5.6.0-0.rc6.git0.1.vanilla.knurd.1.fc31.x86_64/build/arch/x86/include/asm/page.h:12:
/lib/modules/5.6.0-0.rc6.git0.1.vanilla.knurd.1.fc31.x86_64/build/arch/x86/include/asm/page_64.h:49:2:
error: expected '(' after 'asm'
alternative_call_2(clear_page_orig,
Ming Lei <tom.leiming@xxxxxxxxx> 于2020年3月16日周一 下午11:22写道：
>
> On Sun, Mar 15, 2020 at 9:34 AM Feng Li <lifeng1519@xxxxxxxxx> wrote:
> >
> > Hi Ming,
> > This is my cmd to run qemu:
> > qemu-2.12.0/x86_64-softmmu/qemu-system-x86_64 -enable-kvm -device
> > virtio-balloon -cpu host -smp 4 -m 2G -drive
> > file=/root/html/fedora-10g.img,format=raw,cache=none,aio=native,if=none,id=drive-virtio-disk1
> > -device virtio-blk-pci,scsi=off,drive=drive-virtio-disk1,id=virtio-disk1,bootindex=1
> > -drive file=/dev/sdb,format=raw,cache=none,aio=native,if=none,id=drive-virtio-disk2
> > -device virtio-blk-pci,scsi=off,drive=drive-virtio-disk2,id=virtio-disk2,bootindex=2
> > -device virtio-net,netdev=nw1,mac=00:11:22:EE:EE:10 -netdev
> > tap,id=nw1,script=no,downscript=no,ifname=tap0 -serial mon:stdio
> > -nographic -object
> > memory-backend-file,id=mem0,size=2G,mem-path=/dev/hugepages,share=on
> > -numa node,memdev=mem0 -vnc 0.0.0.0:100 -machine usb=on,nvdimm -device
> > usb-tablet -monitor unix:///tmp/a.socket,server,nowait -qmp
> > tcp:0.0.0.0:2234,server,nowait
> >
> > OS image is Fedora 31. Kernel is 5.3.7-301.fc31.x86_64.
> >
> > The address from virio in qemu like this:
> > ========= size: 262144, iovcnt: 64
> > 0: size: 4096 addr: 0x7fffc83f1000
> > 1: size: 4096 addr: 0x7fffc8037000
> > 2: size: 4096 addr: 0x7fffd3710000
> > 3: size: 4096 addr: 0x7fffd5624000
> > 4: size: 4096 addr: 0x7fffc766c000
> > 5: size: 4096 addr: 0x7fffc7c21000
> > 6: size: 4096 addr: 0x7fffc8d54000
> > 7: size: 4096 addr: 0x7fffc8fc6000
> > 8: size: 4096 addr: 0x7fffd5659000
> > 9: size: 4096 addr: 0x7fffc7f88000
> > 10: size: 4096 addr: 0x7fffc767b000
> > 11: size: 4096 addr: 0x7fffc8332000
> > 12: size: 4096 addr: 0x7fffb4297000
> > 13: size: 4096 addr: 0x7fffc8888000
> > 14: size: 4096 addr: 0x7fffc93d7000
> > 15: size: 4096 addr: 0x7fffc9f1f000
> >
> > They are not contiguous pages, so the pages in bvec are not continus
> > physical pages.
> >
> > I don't know how to dump the bvec address in bio without recompiling the kernel.
>
> I just run similar test on 5.3.11-100.fc29.x86_64, and the observation
> is similar with
> yours.
>
> However, not observe similar problem in 5.6-rc kernel in VM, maybe kernel config
> causes the difference.
>
> BTW, I usually use the attached bcc script to observe bvec pages, and you may
> try that on upstream kernel.
>
> Thanks,
> Ming
