On Mon, 2023-02-13 at 10:23 +0800, Wentong Wu wrote: > v2 without change log, why? > Intel Visual Sensing Controller (IVSC), codenamed "Clover Falls", is > a > companion chip designed to provide secure and low power vision > capability > to IA platforms. IVSC is available in existing commercial platforms > from > multiple OEMs. > > The primary use case of IVSC is to bring in context awareness. IVSC > interfaces directly with the platform main camera sensor via a CSI-2 > link > and processes the image data with the embedded AI engine. The > detected > events are sent over I2C to ISH (Intel Sensor Hub) for additional > data > fusion from multiple sensors. The fusion results are used to > implement > advanced use cases like: > - Face detection to unlock screen > - Detect user presence to manage backlight setting or waking up > system > > Since the Image Processing Unit(IPU) used on the host processor needs > to > configure the CSI-2 link in normal camera usages, the CSI-2 link and > camera sensor can only be used in mutually-exclusive ways by host IPU > and > IVSC. By default the IVSC owns the CSI-2 link and camera sensor. The > IPU > driver can take ownership of the CSI-2 link and camera sensor using > interfaces provided by this IVSC driver. > > Switching ownership requires an interface with two different hardware > modules inside IVSC. The software interface to these modules is via > Intel > MEI (The Intel Management Engine) commands. These two hardware > modules > have two different MEI UUIDs to enumerate. These hardware modules > are: > - ACE (Algorithm Context Engine): This module is for algorithm > computing > when IVSC owns camera sensor. Also ACE module controls camera > sensor's > ownership. This hardware module is used to set ownership of camera > sensor. > - CSI (Camera Serial Interface): This module is used to route camera > sensor data either to IVSC or to host for IPU driver and application. > > IVSC also provides a privacy mode. When privacy mode is turned on, > camera sensor can't be used. This means that both ACE and host IPU > can't > get image data. And when this mode is turned on, host IPU driver is > informed via a registered callback, so that user can be notified. > > In summary, to acquire ownership of camera by IPU driver, first ACE > module needs to be informed of ownership and then to setup MIPI CSI-2 > link for the camera sensor and IPU. > > Implementation: > There are two different drivers to handle ACE and CSI hardware > modules > inside IVSC. > - mei_csi: MEI client driver to send commands and receive > notifications > from CSI module. > - mei_ace: MEI client driver to send commands and get status from > ACE > module. > Interface is exposed via ivsc.h to acquire and release camera sensor > and > CSI-2 link. > > Below diagram shows connections of IVSC/ISH/IPU/Camera sensor. > --------------------------------------------------------------------- > -------- > > Host > > Processor > > | > > > > | > > ----------------- ----------------- ------------- > > -- | > > | | | | | > > | I2C | > > | IPU | | ISH | |camera > > driver|--| | > > | | | | | > > | | | > > ----------------- ----------------- ------------- > > -- | | > > | | > > | | | > > | | ------------- > > -- | | > > | | | > > | | | > > | | | IVSC driver > > | | | > > | | | > > | | | > > | | ------------- > > -- | | > > | | > > | | | > ----------------|-----------------------|----------------------|----- > ----|--- > | CSI | I2C > |SPI | > | | > | | > ----------------|-----------------------|---------------- > | | > > IVSC | | > > | | > > | | > > | | > > ----------------- ----------------- | > > | | > > | | | | | > > | | > > | CSI | | ACE | |------ > > | | > > | | | | > > | | > > ----------------- ----------------- > > | | > > | | I2C > > | | > ----------------|-----------------------|---------------- > | > | CSI > | | > | > | | > -------------------------------- > | > | | > I2C | > | camera sensor |------------------------- > ----| > | | > -------------------------------- > > Wentong Wu (3): > media: pci: intel: ivsc: Add CSI submodule > media: pci: intel: ivsc: Add ACE submodule > media: pci: intel: ivsc: Add acquire/release API for ivsc > > drivers/media/pci/Kconfig | 1 + > drivers/media/pci/intel/Makefile | 2 + > drivers/media/pci/intel/ivsc/Kconfig | 12 + > drivers/media/pci/intel/ivsc/Makefile | 7 + > drivers/media/pci/intel/ivsc/ivsc.c | 84 +++++ > drivers/media/pci/intel/ivsc/mei_ace.c | 472 > +++++++++++++++++++++++++ > drivers/media/pci/intel/ivsc/mei_ace.h | 36 ++ > drivers/media/pci/intel/ivsc/mei_csi.c | 342 ++++++++++++++++++ > drivers/media/pci/intel/ivsc/mei_csi.h | 60 ++++ > include/linux/ivsc.h | 74 ++++ > 10 files changed, 1090 insertions(+) > create mode 100644 drivers/media/pci/intel/ivsc/Kconfig > create mode 100644 drivers/media/pci/intel/ivsc/Makefile > create mode 100644 drivers/media/pci/intel/ivsc/ivsc.c > create mode 100644 drivers/media/pci/intel/ivsc/mei_ace.c > create mode 100644 drivers/media/pci/intel/ivsc/mei_ace.h > create mode 100644 drivers/media/pci/intel/ivsc/mei_csi.c > create mode 100644 drivers/media/pci/intel/ivsc/mei_csi.h > create mode 100644 include/linux/ivsc.h >
