This patchset implements the core part of Intel GVT-g [1] for graphics virtualization. The patchset consists of two subsets: One is the device model implementation of vGPUs for virtual machines (patch 09 - 29) Another is the needed i915 native driver changes for that implementation (patch 01 - 08). We hope to get feedbacks of the native driver changes from i915 developers, while the device model part can be a reference to see how we utilize those changes. Background ---------- Back to 1 year ago, we ever sent RFC of iGVT-g changes to native i915 drivers, so called "host mediation"[2]. The architecture is that host i915 driver runs on top of iGVT-g device model, and device model controls hardware directly if it is enabled. After discussion, we agreed an architecture change to let native driver on host controls hardware directly, and integrate virtualization changes into i915. Last year we followed this direction to remove host mediation, and move some virtualization related logics into i915. For instance, the interrupt part [3]. Later we found that the i915 changes can be much reduced if we virtualize more resources for VMs. Now we are happy to see that the i915 changes are very limited. Major i915 changes ------------------ In the new architecture, device-model works as a client. We do not mediate any i915 accesses of hardware resources. VM's vGPU resources keep virtual as much as possible. One example is interrupt. vGPU's interrupt will be triggered either through virtual timer (vblnk) or workload status changes (ctx_switch), not directly bound to hardware interrupts. Hence we do not need to touch i915 interrupt framework at all. The remaining i915 changes are: - vGPU guest framebuffer GEM object support Wrap a guest framebuffer into a host i915 GEM object. So host graphics application could be able to access guest display surface easily via DRM/GL APIs. Related commits: drm/i915: Support vGPU guest framebuffer GEM object - Host i915 GGTT graphics memory space/fence registers balloon i915 host owns limited GGTT graphics memory space/fence registers under GVT environment, others are managed by GVT resource allocator, which allocates/de-allocates these resources for GVT guest. Related commits: drm/i915: Introduce host graphics memory balloon - GVT context creation GVT workload scheduler needs special host LRC contexts, so called "shadow LRC context" to submit guest workload to host i915. During the guest workload submission. GVT fills the shadow LRC context with the content guest LRC context: engine context is copied without changes, ring context is mostly owned by host i915, except the PPGTT root pointers, which will be filled with the shadow PPGTT page table root pointers managed by GVT-g. Related commits: drm/i915: gem: Introduce i915_gem_create_gvt_context() drm/i915: Ondemand populate context addressing mode bit drm/i915: Do not initialize the engine state of GVT context drm/i915: Do not populate PPGTT root pointers for GVT context - GVT context scheduling Since GVT guest workload will also be dispatched through i915 GEM submission system, the GVT context mentioned above is also scheduled by host i915 LRC routines. The workload inside a GVT context may come from different GVT guests, some render MMIOs have to be configured with the guest render MMIOs before the GVT context gets kicked out. It's hard to submit a GVT context with a host LRC context in the same ELSP combo. The LRC context submission policy for GVT context is adjusted as: * If ELSP submission routines pick a GVT context, then stop to pick next context. (Single ELSP submission - for easy render MMIOs save/restore) * If ELSP submission routines pick a i915 context, and the next context is a GVT context, then stop pick the next context. (Single ELSP submission - we cannot skip the GVT context, as the seqno of workload in GVT context may be passed, but the workload in GVT context haven't be executed yet) Related commits: drm/i915: GVT context scheduling Difference from community release --------------------------------- This patchset is different from regular iGVT-g code release[4], which is still based on old host-mediated architecture. Furthermore, this patchset only supports BDW whereas code release supports HSW/BDW/SKL. We will add SKL support later based on this RFC code and HSW support will be dropped. Internally we tested this RFC patchset with both linux and windows VM and the architecture changes work fine. Acknowledgement --------------- iGVT-g implementation is several years effort and many people contributed to the code. There names are not here yet. In later formal patchset we will reflect individual's contribution. Meanwhile, in the previous iGVT-g related discussion, Daniel, Chris and Joonas ever gave very good inputs. We appreciate them and look forward to more comments/suggestions from community. We are trying to get more familiar with i915 and willing to adopt suggestions to keep improving. We hope to work with community together to make iGVT-g a great component in i915 to support graphics virtualization. Thanks! Reference --------- [1] https://01.org/igvt-g [2] http://lists.freedesktop.org/archives/intel-gfx/2014-September/053098.html [3] http://lists.freedesktop.org/archives/intel-gfx/2015-September/075397.html [4] http://lists.freedesktop.org/archives/intel-gfx/2016-January/086372.html *** BLURB HERE *** Bing Niu (2): drm/i915: Introduce host graphics memory balloon for gvt drm/i915: gvt: Full display virtualization Niu Bing (1): drm/i915: gvt: vGPU framebuffer format decoder Niu,Bing (1): drm/i915: gvt: vGPU MMIO register emulation Yi Sun (1): drm/i915: gvt: Introduce GVT control interface Yulei Zhang (2): drm/i915: gvt: Resource allocator drm/i915: gvt: vGPU command scanner Zhi Wang (21): drm/i915/gvt: Introduce the basic architecture of GVT-g drm/i915: Introduce GVT context creation API drm/i915: Ondemand populate context addressing mode bit drm/i915: Do not populate PPGTT root pointers for GVT context drm/i915: Do not initialize the engine state of GVT context drm/i915: GVT context scheduling drm/i915: gvt: Basic mmio emulation state drm/i915: gvt: update PVINFO page definition in i915_vgpu.h drm/i915: gvt: vGPU life cycle management drm/i915: gvt: trace stub drm/i915: gvt: vGPU interrupt emulation framework drm/i915: gvt: vGPU graphics memory emulation framework drm/i915: gvt: Generic MPT framework gvt: Xen hypervisor GVT-g MPT module drm/i915: gvt: vGPU configuration emulation drm/i915: gvt: vGPU OpRegion emulation drm/i915: gvt: Full execlist status emulation drm/i915: gvt: vGPU execlist workload submission drm/i915: gvt: workload scheduler drm/i915: gvt: vGPU schedule policy framework drm/i915: gvt: vGPU context switch Zhiyuan Lv (1): drm/i915: Support vGPU guest framebuffer GEM object arch/x86/include/asm/xen/hypercall.h | 7 + arch/x86/include/asm/xen/interface.h | 4 + arch/x86/xen/mmu.c | 83 ++ drivers/gpu/drm/i915/Kconfig | 16 + drivers/gpu/drm/i915/Makefile | 3 + drivers/gpu/drm/i915/gvt/Makefile | 8 + drivers/gpu/drm/i915/gvt/aperture_gm.c | 225 +++ drivers/gpu/drm/i915/gvt/cfg_space.c | 190 +++ drivers/gpu/drm/i915/gvt/cfg_space.h | 33 + drivers/gpu/drm/i915/gvt/cmd_parser.c | 2018 +++++++++++++++++++++++++++ drivers/gpu/drm/i915/gvt/cmd_parser.h | 466 +++++++ drivers/gpu/drm/i915/gvt/control.c | 181 +++ drivers/gpu/drm/i915/gvt/debug.h | 103 ++ drivers/gpu/drm/i915/gvt/display.c | 233 ++++ drivers/gpu/drm/i915/gvt/display.h | 129 ++ drivers/gpu/drm/i915/gvt/edid.c | 493 +++++++ drivers/gpu/drm/i915/gvt/edid.h | 184 +++ drivers/gpu/drm/i915/gvt/execlist.c | 612 +++++++++ drivers/gpu/drm/i915/gvt/execlist.h | 179 +++ drivers/gpu/drm/i915/gvt/fb_decoder.c | 325 +++++ drivers/gpu/drm/i915/gvt/fb_decoder.h | 110 ++ drivers/gpu/drm/i915/gvt/gtt.c | 1912 ++++++++++++++++++++++++++ drivers/gpu/drm/i915/gvt/gtt.h | 305 ++++ drivers/gpu/drm/i915/gvt/gvt.c | 446 ++++++ drivers/gpu/drm/i915/gvt/gvt.h | 715 ++++++++++ drivers/gpu/drm/i915/gvt/handlers.c | 2141 +++++++++++++++++++++++++++++ drivers/gpu/drm/i915/gvt/hypercall.h | 56 + drivers/gpu/drm/i915/gvt/instance.c | 272 ++++ drivers/gpu/drm/i915/gvt/interrupt.c | 721 ++++++++++ drivers/gpu/drm/i915/gvt/interrupt.h | 257 ++++ drivers/gpu/drm/i915/gvt/mmio.c | 545 ++++++++ drivers/gpu/drm/i915/gvt/mmio.h | 105 ++ drivers/gpu/drm/i915/gvt/mpt.h | 169 +++ drivers/gpu/drm/i915/gvt/opregion.c | 314 +++++ drivers/gpu/drm/i915/gvt/opregion.h | 34 + drivers/gpu/drm/i915/gvt/params.c | 54 + drivers/gpu/drm/i915/gvt/params.h | 49 + drivers/gpu/drm/i915/gvt/perf.h | 69 + drivers/gpu/drm/i915/gvt/reg.h | 889 ++++++++++++ drivers/gpu/drm/i915/gvt/render.c | 104 ++ drivers/gpu/drm/i915/gvt/render.h | 31 + drivers/gpu/drm/i915/gvt/sched_policy.c | 295 ++++ drivers/gpu/drm/i915/gvt/sched_policy.h | 48 + drivers/gpu/drm/i915/gvt/scheduler.c | 501 +++++++ drivers/gpu/drm/i915/gvt/scheduler.h | 85 ++ drivers/gpu/drm/i915/gvt/trace.h | 260 ++++ drivers/gpu/drm/i915/gvt/trace_points.c | 29 + drivers/gpu/drm/i915/gvt/utility.c | 91 ++ drivers/gpu/drm/i915/i915_dma.c | 20 + drivers/gpu/drm/i915/i915_drv.h | 29 + drivers/gpu/drm/i915/i915_gem.c | 3 + drivers/gpu/drm/i915/i915_gem_context.c | 38 + drivers/gpu/drm/i915/i915_gem_gtt.c | 4 +- drivers/gpu/drm/i915/i915_gem_gvtbuffer.c | 247 ++++ drivers/gpu/drm/i915/i915_vgpu.c | 16 +- drivers/gpu/drm/i915/i915_vgpu.h | 94 +- drivers/gpu/drm/i915/intel_lrc.c | 64 +- drivers/xen/Kconfig | 5 + drivers/xen/Makefile | 6 + drivers/xen/xengt.c | 1153 ++++++++++++++++ include/uapi/drm/i915_drm.h | 38 + include/xen/interface/hvm/hvm_op.h | 177 ++- include/xen/interface/hvm/ioreq.h | 132 ++ include/xen/interface/memory.h | 28 + include/xen/interface/xen.h | 107 ++ include/xen/xen-ops.h | 5 + 66 files changed, 18222 insertions(+), 43 deletions(-) create mode 100644 drivers/gpu/drm/i915/gvt/Makefile create mode 100644 drivers/gpu/drm/i915/gvt/aperture_gm.c create mode 100644 drivers/gpu/drm/i915/gvt/cfg_space.c create mode 100644 drivers/gpu/drm/i915/gvt/cfg_space.h create mode 100644 drivers/gpu/drm/i915/gvt/cmd_parser.c create mode 100644 drivers/gpu/drm/i915/gvt/cmd_parser.h create mode 100644 drivers/gpu/drm/i915/gvt/control.c create mode 100644 drivers/gpu/drm/i915/gvt/debug.h create mode 100644 drivers/gpu/drm/i915/gvt/display.c create mode 100644 drivers/gpu/drm/i915/gvt/display.h create mode 100644 drivers/gpu/drm/i915/gvt/edid.c create mode 100644 drivers/gpu/drm/i915/gvt/edid.h create mode 100644 drivers/gpu/drm/i915/gvt/execlist.c create mode 100644 drivers/gpu/drm/i915/gvt/execlist.h create mode 100644 drivers/gpu/drm/i915/gvt/fb_decoder.c create mode 100644 drivers/gpu/drm/i915/gvt/fb_decoder.h create mode 100644 drivers/gpu/drm/i915/gvt/gtt.c create mode 100644 drivers/gpu/drm/i915/gvt/gtt.h create mode 100644 drivers/gpu/drm/i915/gvt/gvt.c create mode 100644 drivers/gpu/drm/i915/gvt/gvt.h create mode 100644 drivers/gpu/drm/i915/gvt/handlers.c create mode 100644 drivers/gpu/drm/i915/gvt/hypercall.h create mode 100644 drivers/gpu/drm/i915/gvt/instance.c create mode 100644 drivers/gpu/drm/i915/gvt/interrupt.c create mode 100644 drivers/gpu/drm/i915/gvt/interrupt.h create mode 100644 drivers/gpu/drm/i915/gvt/mmio.c create mode 100644 drivers/gpu/drm/i915/gvt/mmio.h create mode 100644 drivers/gpu/drm/i915/gvt/mpt.h create mode 100644 drivers/gpu/drm/i915/gvt/opregion.c create mode 100644 drivers/gpu/drm/i915/gvt/opregion.h create mode 100644 drivers/gpu/drm/i915/gvt/params.c create mode 100644 drivers/gpu/drm/i915/gvt/params.h create mode 100644 drivers/gpu/drm/i915/gvt/perf.h create mode 100644 drivers/gpu/drm/i915/gvt/reg.h create mode 100644 drivers/gpu/drm/i915/gvt/render.c create mode 100644 drivers/gpu/drm/i915/gvt/render.h create mode 100644 drivers/gpu/drm/i915/gvt/sched_policy.c create mode 100644 drivers/gpu/drm/i915/gvt/sched_policy.h create mode 100644 drivers/gpu/drm/i915/gvt/scheduler.c create mode 100644 drivers/gpu/drm/i915/gvt/scheduler.h create mode 100644 drivers/gpu/drm/i915/gvt/trace.h create mode 100644 drivers/gpu/drm/i915/gvt/trace_points.c create mode 100644 drivers/gpu/drm/i915/gvt/utility.c create mode 100644 drivers/gpu/drm/i915/i915_gem_gvtbuffer.c create mode 100644 drivers/xen/xengt.c create mode 100644 include/xen/interface/hvm/ioreq.h -- 1.9.1 _______________________________________________ Intel-gfx mailing list Intel-gfx@xxxxxxxxxxxxxxxxxxxxx http://lists.freedesktop.org/mailman/listinfo/intel-gfx
