Hi,
I have searched high and low on the internet, but I have not found an
answer. I am hoping someone on this list can help or point me in the
right direction.
My company manufactures test equipment for semiconductor companies. A
typical tester consists of a test head with multiple different kinds of
instruments of our design. Some instruments are designed for high speed
digital stimulus and compare. Some for parametric voltage/current
sense/force (called VI), some for RF, etc.
The test head can be configured to hold up to 40 of these instruments in
just about any combination. The databus connection to the instruments is
a PCIe fabric that takes a 16 lane PCIe slot from a tester workstation
running Linux (currently Centos 7.2), and through a combination of PCIe
switches (such as the Broadcom PLX 8664).
When we first marketed the tester, we used Centos 4, and have progressed
up to the latest Centos 7 with our current software release.
Unfortunately, our customer base does not migrate easily, and this is
something we can't control, so we have to accommodate these customer by
continuing to support the older software releases on the older Linux
versions. This proves difficult with the advent of newer workstations
that can't run with the older Linux versions. While we strive to get
commitments from our workstation vendors (iBase, SuperMicro, etc) that
these workstations have a minimum 5 year life span, at some point, they
become End Of Life and we can no longer obtain them. Yet our customers
still need systems running older versions of Linux.
The idea is to use some kind of virtualization to run the latest Linux
on a more modern workstation, then run the desired Linux on a guest VM.
Since our software requires access to the PCIe devices, we need to use
PCIe pass through to the guest VM. Herein lies the problem. While I can
successfully pass through our PCIe compliant instruments to the guest VM
without problem, I still need to be able to access the PLX PCIe
switches, because in some cases, they too provide a vital function in
the system. However, the KVM architecture disallows the pass through of
bridges and switches to the guest VM. I can understand the bridge/switch
aspect, however, these PLX devices are both a switch and an endpoint.
Their upstream port has a 256K BAR 0 through which many extra functions
are available, including control of virtual switch capability,
non-transparent ports, onboard GPIO pins, some even have general purpose
DMA engines. These are all accessed via BAR 0.
So my question is, how (any ideas welcome) can I access these BAR
resources from within the guest VM? I have full control over the kernel
driver (I wrote it) that our software uses to access the instruments,
including memory mapping, control of DMA engines on our instruments,
etc. so even if I had to modify our driver in some way, that is OK. I
could even customize the driver to run something on the VM host to
support the guest VM, if I had some starting point on how this might be
accomplished.
Thank you for reading this long winded post, but some background is
necessary to understand what I am trying to do.
-----------------------------------------------------
Russ Poffenberger | System Architecture Group
Phone: +1 / 408 / 635-6049
russell.poffenberger@xxxxxxxxxx
Xcerra Corporation, LTX/Credence Division
880 N. McCarthy Blvd, Suite 100
Milpitas, CA 95035, USA
-----------------------------------------------------
