Hi, Please help review this V5 patch set, thanks a lot! BR, Wentong > -----Original Message----- > From: Wu, Wentong <wentong.wu@xxxxxxxxx> > Sent: Monday, April 10, 2023 11:14 AM > To: sakari.ailus@xxxxxxxxxxxxxxx; hdegoede@xxxxxxxxxx; djrscally@xxxxxxxxx; > > Intel Visual Sensing Controller (IVSC), codenamed "Clover Falls", is a companion > chip designed to provide secure and low power vision capability to IA platforms. > IVSC is available in existing commercial platforms from multiple OEMs. > > The primary use case of IVSC is to bring in context awareness. IVSC interfaces > directly with the platform main camera sensor via a CSI-2 link and processes the > image data with the embedded AI engine. The detected events are sent over I2C > to ISH (Intel Sensor Hub) for additional data fusion from multiple sensors. The > fusion results are used to implement advanced use cases like: > - Face detection to unlock screen > - Detect user presence to manage backlight setting or waking up system > > Since the Image Processing Unit(IPU) used on the host processor needs to > configure the CSI-2 link in normal camera usages, the CSI-2 link and camera > sensor can only be used in mutually-exclusive ways by host IPU and IVSC. By > default the IVSC owns the CSI-2 link and camera sensor. The IPU driver can take > ownership of the CSI-2 link and camera sensor using the interfaces exported via > v4l2 sub-device. > > Switching ownership requires an interface with two different hardware modules > inside IVSC. The software interface to these modules is via Intel MEI (The Intel > Management Engine) commands. These two hardware modules have two > different MEI UUIDs to enumerate. These hardware modules are: > - ACE (Algorithm Context Engine): This module is for algorithm computing when > IVSC owns camera sensor. Also ACE module controls camera sensor's ownership. > This hardware module is used to set ownership of camera sensor. > - CSI (Camera Serial Interface): This module is used to route camera sensor data > either to IVSC or to host for IPU driver and application. > > IVSC also provides a privacy mode. When privacy mode is turned on, camera > sensor can't be used. This means that both ACE and host IPU can't get image > data. And when this mode is turned on, users are informed via v4l2 control API. > > In summary, to acquire ownership of camera by IPU driver, first ACE module > needs to be informed of ownership and then to setup MIPI CSI-2 link for the > camera sensor and IPU. > > Implementation: > There are two different drivers to handle ACE and CSI hardware modules inside > IVSC. > - ivsc_csi: MEI client driver to send commands and receive notifications from > CSI module. > - ivsc_ace: MEI client driver to send commands and get status from ACE module. > Interface is exposed via v4l2 sub-devcie APIs to acquire and release camera > sensor and CSI-2 link. > > Below diagram shows connections of IVSC/ISH/IPU/Camera sensor. > ----------------------------------------------------------------------------- > | Host Processor | > | | > | ----------------- ----------------- --------------- | > | | | | | | | I2C | > | | IPU | | ISH | |camera driver|--| | > | | | | | | | | | > | ----------------- ----------------- --------------- | | > | | | | | | > | | | --------------- | | > | | | | | | | > | | | | IVSC driver | | | > | | | | | | | > | | | --------------- | | > | | | | | | > ----------------|-----------------------|----------------------|---------|--- > | CSI | I2C |SPI | > | | | | > ----------------|-----------------------|---------------- | | > | IVSC | | | | > | | | | | > | ----------------- ----------------- | | | > | | | | | | | | > | | CSI | | ACE | |------| | > | | | | | | | > | ----------------- ----------------- | | > | | | I2C | | > ----------------|-----------------------|---------------- | > | CSI | | > | | | > -------------------------------- | > | | I2C | > | camera sensor |-----------------------------| > | | > -------------------------------- > > --- > v5: > - probe mei_csi only after software node has been setup > > v4: > - call v4l2_ctrl_handler_free() if setting up the handler failed > - set V4L2_CTRL_FLAG_READ_ONLY for privacy_ctrl > - add dev_warn if failed to switch CSI-2 link to IVSC > - use v4l2_fwnode_endpoint_alloc_parse to get num_data_lanes > - add document about how sensor connected to IVSC is powered > - move lock to mei_ace_send > - check return value for device_link_add > > Wentong Wu (3): > media: pci: intel: ivsc: Add CSI submodule > media: pci: intel: ivsc: Add ACE submodule > ACPI: delay enumeration of devices with a _DEP pointing to IVSC device > > drivers/acpi/scan.c | 2 + > drivers/media/pci/Kconfig | 1 + > drivers/media/pci/intel/Makefile | 2 + > drivers/media/pci/intel/ivsc/Kconfig | 12 + > drivers/media/pci/intel/ivsc/Makefile | 9 + > drivers/media/pci/intel/ivsc/mei_ace.c | 560 ++++++++++++++++++++++++ > drivers/media/pci/intel/ivsc/mei_csi.c | 772 > +++++++++++++++++++++++++++++++++ > 7 files changed, 1358 insertions(+) > create mode 100644 drivers/media/pci/intel/ivsc/Kconfig > create mode 100644 drivers/media/pci/intel/ivsc/Makefile > create mode 100644 drivers/media/pci/intel/ivsc/mei_ace.c > create mode 100644 drivers/media/pci/intel/ivsc/mei_csi.c > > -- > 2.7.4
