maybe because i modify the code in assigned_dev_iomem_map().
i used RHEL6, and calc_assigned_dev_id is below：
static uint32_t calc_assigned_dev_id(uint8_t bus, uint8_t devfn)
{
return (uint32_t)bus << 8 | (uint32_t)devfn;
}
and in patch there are there param.
+ msix_mmio.id = calc_assigned_dev_id(r_dev->h_segnr,
+ r_dev->h_busnr, r_dev->h_devfn);
#ifdef KVM_CAP_MSIX_MASK
if (cap_mask) {
memset(&msix_mmio, 0, sizeof msix_mmio);
msix_mmio.id = calc_assigned_dev_id(r_dev->h_busnr,
r_dev->h_devfn);
msix_mmio.type = KVM_MSIX_TYPE_ASSIGNED_DEV;
msix_mmio.base_addr = e_phys + offset;
msix_mmio.max_entries_nr = r_dev->max_msix_entries_nr;
msix_mmio.flags = KVM_MSIX_MMIO_FLAG_REGISTER;
ret = kvm_update_msix_mmio(kvm_context, &msix_mmio);
if (ret)
fprintf(stderr, "fail to register in-kernel msix_mmio!\n");
}
#endif
2010/12/1 Yang, Sheng <sheng.yang@xxxxxxxxx>:
> On Wednesday 01 December 2010 16:54:16 lidong chen wrote:
>> yes, i patch qemu as well.
>>
>> and i found the address of second vf is not in mmio range. the first
>> one is fine.
>
> So looks like something wrong with MMIO register part. Could you check the
> registeration in assigned_dev_iomem_map() of the 4th patch for QEmu? I suppose
> something wrong with it. I would try to reproduce it here.
>
> And if you only use one vf, how about the gain?
>
> --
> regards
> Yang, Sheng
>
>>
>> 2010/12/1 Yang, Sheng <sheng.yang@xxxxxxxxx>:
>> > On Wednesday 01 December 2010 16:41:38 lidong chen wrote:
>> >> I used sr-iov, give each vm 2 vf.
>> >> after apply the patch, and i found performence is the same.
>> >>
>> >> the reason is in function msix_mmio_write, mostly addr is not in mmio
>> >> range.
>> >
>> > Did you patch qemu as well? You can see it's impossible for kernel part
>> > to work alone...
>> >
>> > http://www.mail-archive.com/kvm@xxxxxxxxxxxxxxx/msg44368.html
>> >
>> > --
>> > regards
>> > Yang, Sheng
>> >
>> >> static int msix_mmio_write(struct kvm_io_device *this, gpa_t addr, int
>> >> len, const void *val)
>> >> {
>> >> struct kvm_assigned_dev_kernel *adev =
>> >> container_of(this, struct kvm_assigned_dev_kernel,
>> >> msix_mmio_dev);
>> >> int idx, r = 0;
>> >> unsigned long new_val = *(unsigned long *)val;
>> >>
>> >> mutex_lock(&adev->kvm->lock);
>> >> if (!msix_mmio_in_range(adev, addr, len)) {
>> >> // return here.
>> >> r = -EOPNOTSUPP;
>> >> goto out;
>> >> }
>> >>
>> >> i printk the value:
>> >> addr start end len
>> >> F004C00C F0044000 F0044030 4
>> >>
>> >> 00:06.0 Ethernet controller: Intel Corporation Unknown device 10ed (rev
>> >> 01) Subsystem: Intel Corporation Unknown device 000c
>> >> Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
>> >> ParErr- Stepping- SERR- FastB2B-
>> >> Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
>> >> <TAbort- <MAbort- >SERR- <PERR-
>> >> Latency: 0
>> >> Region 0: Memory at f0040000 (32-bit, non-prefetchable) [size=16K]
>> >> Region 3: Memory at f0044000 (32-bit, non-prefetchable) [size=16K]
>> >> Capabilities: [40] MSI-X: Enable+ Mask- TabSize=3
>> >> Vector table: BAR=3 offset=00000000
>> >> PBA: BAR=3 offset=00002000
>> >>
>> >> 00:07.0 Ethernet controller: Intel Corporation Unknown device 10ed (rev
>> >> 01) Subsystem: Intel Corporation Unknown device 000c
>> >> Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
>> >> ParErr- Stepping- SERR- FastB2B-
>> >> Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
>> >> <TAbort- <MAbort- >SERR- <PERR-
>> >> Latency: 0
>> >> Region 0: Memory at f0048000 (32-bit, non-prefetchable) [size=16K]
>> >> Region 3: Memory at f004c000 (32-bit, non-prefetchable) [size=16K]
>> >> Capabilities: [40] MSI-X: Enable+ Mask- TabSize=3
>> >> Vector table: BAR=3 offset=00000000
>> >> PBA: BAR=3 offset=00002000
>> >>
>> >>
>> >>
>> >> +static bool msix_mmio_in_range(struct kvm_assigned_dev_kernel *adev,
>> >> + gpa_t addr, int len)
>> >> +{
>> >> + gpa_t start, end;
>> >> +
>> >> + BUG_ON(adev->msix_mmio_base == 0);
>> >> + start = adev->msix_mmio_base;
>> >> + end = adev->msix_mmio_base + PCI_MSIX_ENTRY_SIZE *
>> >> + adev->msix_max_entries_nr;
>> >> + if (addr >= start && addr + len <= end)
>> >> + return true;
>> >> +
>> >> + return false;
>> >> +}
>> >>
>> >> 2010/11/30 Yang, Sheng <sheng.yang@xxxxxxxxx>:
>> >> > On Tuesday 30 November 2010 17:10:11 lidong chen wrote:
>> >> >> sr-iov also meet this problem, MSIX mask waste a lot of cpu resource.
>> >> >>
>> >> >> I test kvm with sriov, which the vf driver could not disable msix.
>> >> >> so the host os waste a lot of cpu. cpu rate of host os is 90%.
>> >> >>
>> >> >> then I test xen with sriov, there ara also a lot of vm exits caused
>> >> >> by MSIX mask.
>> >> >> but the cpu rate of xen and domain0 is less than kvm. cpu rate of xen
>> >> >> and domain0 is 60%.
>> >> >>
>> >> >> without sr-iov, the cpu rate of xen and domain0 is higher than kvm.
>> >> >>
>> >> >> so i think the problem is kvm waste more cpu resource to deal with
>> >> >> MSIX mask. and we can see how xen deal with MSIX mask.
>> >> >>
>> >> >> if this problem sloved, maybe with MSIX enabled, the performace is
>> >> >> better.
>> >> >
>> >> > Please refer to my posted patches for this issue.
>> >> >
>> >> > http://www.spinics.net/lists/kvm/msg44992.html
>> >> >
>> >> > --
>> >> > regards
>> >> > Yang, Sheng
>> >> >
>> >> >> 2010/11/23 Avi Kivity <avi@xxxxxxxxxx>:
>> >> >> > On 11/23/2010 09:27 AM, lidong chen wrote:
>> >> >> >> can you tell me something about this problem.
>> >> >> >> thanks.
>> >> >> >
>> >> >> > Which problem?
>> >> >> >
>> >> >> > --
>> >> >> > I have a truly marvellous patch that fixes the bug which this
>> >> >> > signature is too narrow to contain.
>
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
