Hi Matt, et al,
(Apologies for sending this out right before the holiday; but I have
this in a state where it's unlikely to change further.)
(@Heiko, @Vasily, @Alexander: As you might notice in the diffstat,
this is mostly within drivers/s390/cio/vfio_ccw* with the exception
of one change to idals.h (patch 12). I hope that's okay.)
Here is the first batch of the vfio-ccw channel program handler rework,
which I'm referring to as "the IDA code." Most of it is rearrangement to
make it more readable, and to remove restrictions in place since
commit 0a19e61e6d4c ("vfio: ccw: introduce channel program interfaces").
My hope is that with this off the plate, it will be easier to extend the
vfio-ccw channel program handler for newer, more modern, features.
Some background:
A Format-1 Channel Command Word (CCW) contains a 31-bit data address,
meaning any I/O transfer is limited to the first 2GB of memory.
The concept of an Indirect Data Address (IDA) was introduced long ago
to allow for non-contiguous memory to be used for data transfers,
while still using 31-bit data addresses.
The initial z/Architecture extended the definition of ESA/390's IDA
concept to include a new IDA format that allows for 64-bit data
addresses [1]. The result is three distinct IDA flavors:
- Format-1 IDA (31-bit addresses), each IDAW moves up to 2K of data
- Format-2 IDA (64-bit addresses), each IDAW moves up to 2K of data
- Format-2 IDA (64-bit addresses), each IDAW moves up to 4K of data
The selection of these three possibilities is done by bits within the
Operation-Request Block (ORB), though it should not be a surprise
that the last one is far-and-away the most common these days.
Who would operate on 2K at a time?
While newer features can be masked off (by a CPU model or equivalent),
and guarded by a defensive check in a driver (such as our check for
a Transport Mode ORB in fsm_io_request()), all three of these
possibilities are available by the base z/Architecture, and cannot
be "hidden." So while it might be unlikely for a guest to issue
such an I/O, it's not impossible.
vfio-ccw was written to only support the third of these options,
while the first two are rejected by a check buried in
ccwchain_calc_length(). While a Transport Mode ORB gets a
distinct message logged, no such announcement as to the cause
of the problem is done here, except for a generic -EOPNOTSUPP
return code in the prefetch codepath.
The goal of this series is to rework the channel program handler
such that any of the above IDA formats can be processed and sent
to the hardware. Any Format-1 IDA issued by a guest will be
converted to the 2K Format-2 variety, such that it is able to
use the full 64-bit addressing. This is a change from today,
where any direct-addressed CCW is converted to a 4K Format-2 IDA.
This needs to become a 2K Format-2 IDA to maintain compatibility
with potential IDAs in other CCWs in a chain.
The first few patches at the start of this series are improvements
that could stand alone, regardless of the rework that follows.
The remainder of the series is intended to do the code movement
that would enable these older IDA formats, but the restriction
itself isn't removed until the end.
While I developed this on top of the -parent code that you've
seen recently [2], it isn't explicitly dependent on it and should
be usable/reviewable without it. I -did- base this series on the
addressing fixes that were discussed more recently [3], since
there are intersections with this file in general.
Patch 2 addresses the suggestion you made in that series [4].
Thanks in advance, I look forward to the feedback...
Eric
[1] See most recent Principles of Operation, page 1-6
[2] https://lore.kernel.org/kvm/20221104142007.1314999-1-farman@xxxxxxxxxxxxx/
[3] https://lore.kernel.org/linux-s390/20221121165836.283781-1-farman@xxxxxxxxxxxxx/
[4] https://lore.kernel.org/linux-s390/c9e7229e-a88d-2185-bb6b-a94e9dac7b7a@xxxxxxxxxxxxx/
Eric Farman (16):
vfio/ccw: cleanup some of the mdev commentary
vfio/ccw: simplify the cp_get_orb interface
vfio/ccw: allow non-zero storage keys
vfio/ccw: move where IDA flag is set in ORB
vfio/ccw: replace copy_from_iova with vfio_dma_rw
vfio/ccw: simplify CCW chain fetch routines
vfio/ccw: remove unnecessary malloc alignment
vfio/ccw: pass page count to page_array struct
vfio/ccw: populate page_array struct inline
vfio/ccw: refactor the idaw counter
vfio/ccw: discard second fmt-1 IDAW
vfio/ccw: calculate number of IDAWs regardless of format
vfio/ccw: allocate/populate the guest idal
vfio/ccw: handle a guest Format-1 IDAL
vfio/ccw: don't group contiguous pages on 2K IDAWs
vfio/ccw: remove old IDA format restrictions
Documentation/s390/vfio-ccw.rst | 4 +-
arch/s390/include/asm/idals.h | 12 ++
drivers/s390/cio/vfio_ccw_cp.c | 346 +++++++++++++++++---------------
drivers/s390/cio/vfio_ccw_cp.h | 3 +-
drivers/s390/cio/vfio_ccw_fsm.c | 2 +-
5 files changed, 197 insertions(+), 170 deletions(-)
--
2.34.1
