Signed-off-by: James (Qian) Wang <james.qian.wang@xxxxxxx> --- Documentation/gpu/drivers.rst | 1 + Documentation/gpu/komeda-kms.rst | 483 +++++++++++++++++++++++++++++++ 2 files changed, 484 insertions(+) create mode 100644 Documentation/gpu/komeda-kms.rst diff --git a/Documentation/gpu/drivers.rst b/Documentation/gpu/drivers.rst index 7c1672118a73..978e6da9bbff 100644 --- a/Documentation/gpu/drivers.rst +++ b/Documentation/gpu/drivers.rst @@ -17,6 +17,7 @@ GPU Driver Documentation vkms bridge/dw-hdmi xen-front + komeda-kms .. only:: subproject and html diff --git a/Documentation/gpu/komeda-kms.rst b/Documentation/gpu/komeda-kms.rst new file mode 100644 index 000000000000..8af925ca0869 --- /dev/null +++ b/Documentation/gpu/komeda-kms.rst @@ -0,0 +1,483 @@ +.. SPDX-License-Identifier: GPL-2.0 + +============================== + drm/komeda ARM display driver +============================== + +The drm/komeda driver supports for the ARM display processor D71 and later +IPs, this document is for giving a brief overview of driver design: how it +works and why design it like that. + +Overview of D71 like display IPs +================================ + +From D71, Arm display IP begins to adopt a flexible and modularized +architecture. A display pipeline is made up of multiple individual and +functional pipeline stages called components, and every component has some +specific capabilities that can give the flowed pipeline pixel data a +particular processing. + +Typical D71 components: + +Layer +----- +Layer is the first pipeline stage, which is for preparing the pixel data +for the next stage. It fetches the pixel from memory, decodes it if it's +AFBC, rotates the source image, unpacks or converts YUV pixels to the device +internal RGB pixels, then adjust the color_space of pixels if need. + +Scaler +------ +As its name, scaler is taking responsability for scaling, and D71 also +supports image enhancements by scaler. +The usage of scaler is very flexible and can be connected to layer output +for layer scaling, or connected to compositor and scale the whole display +frame and then feed the output data into wb_layer which will then write it +into memory. + +Compositor (compiz) +------------------- +Compositor is for blending multiple layers or pixel data flows into one +single display frame, and its output frame can be fed into post image +processor and then on the monitor or fed into wb_layer and written to memory +at the same time. And user also can insert a scaler between compositor and +wb_layer to down scale the display frame first and then writing to memory. + +Writeback Layer (wb_layer) +-------------------------- +Writeback layer do the opposite things of Layer, Which connect to compiz and try +to write the composition result to memory. + +Post image processor (improc) +----------------------------- +Post image processor is for adjusting frame data like gamma and color space +to fit the requirements of the monitor. + +Timing controller (timing_ctrlr) +-------------------------------- +Final stage of display pipeline, Timing controller is not for the pixel +handling, but only for controlling the display timing. + +Merger +------ +D71 scaler mostly only has half the horizontal input/output capabilities compare +with Layer, Like if Layer supports 4K input size, the scaler only can supports +2K input/output in some time. To achieve the fully frame scaling, D71 introduce +Layer Split, which split the whole image to two half part and feed them to two +Layers A and B, and do the scaling independently, after scaling the result need +to be feed to merger to merge two part image, and then output to compiz. + +Splitter +-------- +Similar to Layer Split, but Splitter is used for writeback, which split the +compiz result to two part and then feed them to two scaler. + +Possible D71 Pipeline usage +=========================== + +Benefit from the modularized architecture, D71 pipelines can be easily adjusted +to fit different usages, following are some typical pipeline data flow +configurations: + +Single pipeline data flow +------------------------- + +.. kernel-render:: DOT + :alt: Single pipeline digraph + :caption: Single pipeline data flow + + digraph single_ppl { + rankdir=LR; + + subgraph { + "Memory"; + "Monitor"; + } + + subgraph cluster_pipeline { + style=dashed + node [shape=box] + { + node [bgcolor=grey style=dashed] + "Scaler-0"; + "Scaler-1"; + "Scaler-0/1" + } + + node [bgcolor=grey style=filled] + "Layer-0" -> "Scaler-0" + "Layer-1" -> "Scaler-0" + "Layer-2" -> "Scaler-1" + "Layer-3" -> "Scaler-1" + + "Layer-0" -> "Compiz" + "Layer-1" -> "Compiz" + "Layer-2" -> "Compiz" + "Layer-3" -> "Compiz" + "Scaler-0" -> "Compiz" + "Scaler-1" -> "Compiz" + + "Compiz" -> "Scaler-0/1" -> "Wb_layer" + "Compiz" -> "Improc" -> "Timing Controller" + } + + "Wb_layer" -> "Memory" + "Timing Controller" -> "Monitor" + } + +Dual pipeline with Slave enabled +-------------------------------- + +If pipeline_B is free, D71 supports redirect its compositor output to +pipeline_A as an input of compositor of pipeline_A, then pipeline_B doesn't +have its output and work as a slave of pipeline_A. +NOTE: Since the compiz component doesn't output alpha value, the slave pipeline +only can be used for bottom layers composition. + +.. kernel-render:: DOT + :alt: Slave pipeline digraph + :caption: Slave pipeline enabled data flow + + digraph slave_ppl { + rankdir=LR; + + subgraph { + "Memory"; + "Monitor"; + } + node [shape=box] + subgraph cluster_pipeline_slave { + style=dashed + label="Slave Pipeline_B" + node [shape=box] + { + node [bgcolor=grey style=dashed] + "Slave.Scaler-0"; + "Slave.Scaler-1"; + } + + node [bgcolor=grey style=filled] + "Slave.Layer-0" -> "Slave.Scaler-0" + "Slave.Layer-1" -> "Slave.Scaler-0" + "Slave.Layer-2" -> "Slave.Scaler-1" + "Slave.Layer-3" -> "Slave.Scaler-1" + + "Slave.Layer-0" -> "Slave.Compiz" + "Slave.Layer-1" -> "Slave.Compiz" + "Slave.Layer-2" -> "Slave.Compiz" + "Slave.Layer-3" -> "Slave.Compiz" + "Slave.Scaler-0" -> "Slave.Compiz" + "Slave.Scaler-1" -> "Slave.Compiz" + } + + subgraph cluster_pipeline_master { + style=dashed + label="Master Pipeline_A" + node [shape=box] + { + node [bgcolor=grey style=dashed] + "Scaler-0"; + "Scaler-1"; + "Scaler-0/1" + } + + node [bgcolor=grey style=filled] + "Layer-0" -> "Scaler-0" + "Layer-1" -> "Scaler-0" + "Layer-2" -> "Scaler-1" + "Layer-3" -> "Scaler-1" + + "Slave.Compiz" -> "Compiz" + "Layer-0" -> "Compiz" + "Layer-1" -> "Compiz" + "Layer-2" -> "Compiz" + "Layer-3" -> "Compiz" + "Scaler-0" -> "Compiz" + "Scaler-1" -> "Compiz" + + "Compiz" -> "Scaler-0/1" -> "Wb_layer" + "Compiz" -> "Improc" -> "Timing Controller" + } + + "Wb_layer" -> "Memory" + "Timing Controller" -> "Monitor" + } + +Sub-pipelines for input and output +---------------------------------- + +A complete display pipeline can be easily dividied into three sub-pipelines +according to the in/out usage. + +Layer(input) pipeline +~~~~~~~~~~~~~~~~~~~~~ + +.. kernel-render:: DOT + :alt: Layer data digraph + :caption: Layer (input) data flow + + digraph layer_data_flow { + rankdir=LR; + node [shape=box] + + { + node [bgcolor=grey style=dashed] + "Scaler-n"; + } + + "Layer-n" -> "Scaler-n" -> "Compiz" + } + +.. kernel-render:: DOT + :alt: Layer Split digraph + :caption: Layer Split pipeline + + digraph layer_data_flow { + rankdir=LR; + node [shape=box] + + "Layer-0/1" -> "Scaler-0" -> "Merger" + "Layer-2/3" -> "Scaler-1" -> "Merger" + "Merger" -> "Compiz" + } + +Writeback(output) pipeline +~~~~~~~~~~~~~~~~~~~~~~~~~~ +.. kernel-render:: DOT + :alt: writeback digraph + :caption: Writeback(output) data flow + + digraph writeback_data_flow { + rankdir=LR; + node [shape=box] + + { + node [bgcolor=grey style=dashed] + "Scaler-n"; + } + + "Compiz" -> "Scaler-n" -> "Wb_layer" + } + +.. kernel-render:: DOT + :alt: split writeback digraph + :caption: Writeback(output) Split data flow + + digraph writeback_data_flow { + rankdir=LR; + node [shape=box] + + "Compiz" -> "Splitter" + "Splitter" -> "Scaler-0" -> "Merger" + "Splitter" -> "Scaler-1" -> "Merger" + "Merger" -> "Wb_layer" + } + +Display output pipeline +~~~~~~~~~~~~~~~~~~~~~~~ +.. kernel-render:: DOT + :alt: display digraph + :caption: display output data flow + + digraph single_ppl { + rankdir=LR; + node [shape=box] + + "Compiz" -> "Improc" -> "Timing Controller" + } + +In the following section we'll see these three sub-pipelines will be handled +by KMS-plane/wb_conn/crtc respectively. + +Komeda Resource abstraction +=========================== + +struct komeda_pipeline/component +-------------------------------- + +To fully utilize and easily access/configure the HW, driver side also use a +similar architecture: Pipeline/Component to describe the HW features and +capabilities. and a specific component includes two part: + +- Data flow controlling. +- Specific component capabilities and features. + +So driver defines common header struct komeda_component to describe the data +flow controlling. and all specific component are subclass of this base structure. + +.. kernel-doc:: drivers/gpu/drm/arm/display/komeda/komeda_pipeline.h + :internal: + +Resource discovery and initialization +===================================== + +Pipeline and component are used to describe how to handle the pixel data. +We still need a @struct komeda_dev to describe the whole view of the device, and +the control-abilites of device. + +Now we have &komeda_dev, &komeda_pipeline, &komeda_component, but how to fill dev +with pipelines. Since komeda is not only for D71 but also intended for later IPs, +of course we’d better share as much as possible between different IPs, to +achieve it split komeda device into two layers: CORE and CHIP. + +- CORE: for common features and capabilities handling. +- CHIP: for register program and HW specific feature (limitition) handling. + +CORE access to CHIP by three chip func: + +- struct komeda_dev_funcs +- struct komeda_pipeline_funcs +- struct komeda_component_funcs + +.. kernel-doc:: drivers/gpu/drm/arm/display/komeda/komeda_dev.h + :internal: + +Format handling +=============== + +.. kernel-doc:: drivers/gpu/drm/arm/display/komeda/komeda_format_caps.h + :internal: +.. kernel-doc:: drivers/gpu/drm/arm/display/komeda/komeda_framebuffer.h + :internal: + +Attach komeda_dev to DRM-KMS +============================ + +Komeda abstracts resources by pipeline/component, but DRM-KMS uses +crtc/plane/connector. one KMS-obj can not represent only one single component, +since the requirements of a single KMS object cannot simply be achieved by a +single component, usually that needs multiple components to fit the requirement. +Like set mode, gamma, ctm for KMS all target on CRTC-obj, but komeda needs +compiz, improc and timing_ctrlr to work together to fit these requirements. +And a KMS-Plane may requires multiple komeda resources: layer/scaler/compiz. + +So, One KMS-Obj represents a sub-pipeline of komeda resources. + +- Plane: `Layer(input) pipeline`_ +- Wb_connector: `Writeback(output) pipeline`_ +- Crtc: `Display output pipeline`_ + +So, for komeda, we treat KMS crtc/plane/connector as users of pipeline and +component, and one time a pipeline/component only can be used by one user. +And pipeline/component will be treated as private object of DRM-KMS, the state +will be managed drm_atomic_state as well. + +How to map plane to Layer(input) pipeline +----------------------------------------- + +Komeda has mutiple Layer input pipelines, see: +- `Single pipeline data flow`_ +- `Dual pipeline with Slave enabled`_ + +the easist way is binding a plane to a fixed Layer pipeline. but consider the +komeda capabilities: + +- Layer Split, See `Layer(input) pipeline`_ + + Which actually handles a image by two Layer pipelines. if one plane only + represents a fixed Layer, then the Layer_Split need to be handled in user + mode, for configure the pipeline as Layer_Split, user need to so much + details of HW, and pass down lots of private properties. + But if handle it in kernel the logic will be smooth and simple. + +- Slave pipeline, See `Dual pipeline with Slave enabled`_ + + Since the compiz component doesn't output alpha value, the slave pipeline + only can be used for bottom layers composition, komeda driver wants to hide + this limitation to user, the way is pickup suitable Layer according to + Plane->zpos. + + Slave pipeline capabilities of komeda also means the plane in komeda is CRTC + shareable. assign real Layer to plane in kernal according to the real HW + usage and state can also easy the user driver for the sharable plane. + +So for komeda, the KMS-plane doesn't represent a fixed komeda layer pipeline, +but mutiple Layers with same capabilities, komeda will select one or more Layers +to fit the requirement of one KMS-plane. + +Make component/pipeline to be drm_private_obj +--------------------------------------------- + +Add :c:type:`drm_private_obj` to :c:type:`komeda_component`, :c:type:`komeda_pipeline` + +.. code-block:: c + + struct komeda_component { + struct drm_private_obj obj; + … + } + + struct komeda_pipeline { + struct drm_private_obj obj; + … + } + +Tracking component_state/pipeline_state by drm_atomic_state +----------------------------------------------------------- + +Add :c:type:`drm_private_state` and user to :c:type:`komeda_component_state`, +:c:type:`komeda_pipeline_state` + +.. code-block:: c + + struct komeda_component_state { + struct drm_private_state obj; + void *binding_user; + … + } + + struct komeda_pipeline_state { + struct drm_private_state obj; + struct drm_crtc *crtc; + ... + } + +komeda component validation +--------------------------- + +Komeda has multiple types of components, but the process of validation are +similar, usually include following steps: + +.. code-block:: c + + int komeda_xxxx_validate(struct komeda_component_xxx xxx_comp, + struct komeda_component_output *input_dflow, + struct drm_plane/crtc/connector *user, + struct drm_plane/crtc/connector_state, *user_state) + { + setup 1: check if component is needed, like the scaler is optional depend + on the user_state, if unneeded, just return, the caller will put + the data flow into next stage. + Setup 2: check user_state with component features and capabilities to see + if can meet, if not return fail. + Setup 3: get component_state from drm_atomic_state, and try set user to + component, fail if component has been assigned to another user already. + Setup 3: configure the component_state, like set its input component, + convert user_state to component specific state. + Setup 4: adjust the input_dflow and prepare it for the next stage. + } + +komeda_kms Abstraction +---------------------- + +.. kernel-doc:: drivers/gpu/drm/arm/display/komeda/komeda_kms.h + :internal: + +komde_kms Functions +------------------- +.. kernel-doc:: drivers/gpu/drm/arm/display/komeda/komeda_crtc.c + :internal: +.. kernel-doc:: drivers/gpu/drm/arm/display/komeda/komeda_plane.c + :internal: + +Build komeda to be a linux module driver +======================================== + +Now we have two devices: + +- komeda_dev is for describing the real display hardware. +- komeda_kms_dev is for attaching or connecting komeda_dev to DRM-KMS. + +all komeda operations are supplied or operated by komeda_dev or komeda_kms_dev, +the module driver is only a simple wrapper to pass the linux command +(probe/remove/pm) into komeda_dev or komeda_kms_dev. -- 2.17.1