Hi Wentong, On Mon, Mar 27, 2023 at 07:33:48AM +0000, Wu, Wentong wrote: > > > > -----Original Message----- > > From: Sakari Ailus <sakari.ailus@xxxxxxxxxxxxxxx> > > Sent: Monday, March 27, 2023 3:21 PM > > > > Hi Wentong, > > > > On Mon, Mar 27, 2023 at 02:23:05PM +0800, Wentong Wu wrote: > > > Intel Visual Sensing Controller (IVSC), codenamed "Clover Falls", is a > > > companion chip designed to provide secure and low power vision > > > capability to IA platforms. IVSC is available in existing commercial > > > platforms from multiple OEMs. > > > > > > The primary use case of IVSC is to bring in context awareness. IVSC > > > interfaces directly with the platform main camera sensor via a CSI-2 > > > link and processes the image data with the embedded AI engine. The > > > detected events are sent over I2C to ISH (Intel Sensor Hub) for > > > additional data fusion from multiple sensors. The fusion results are > > > used to implement advanced use cases like: > > > - Face detection to unlock screen > > > - Detect user presence to manage backlight setting or waking up > > > system > > > > > > Since the Image Processing Unit(IPU) used on the host processor needs > > > to configure the CSI-2 link in normal camera usages, the CSI-2 link > > > and camera sensor can only be used in mutually-exclusive ways by host > > > IPU and IVSC. By default the IVSC owns the CSI-2 link and camera > > > sensor. The IPU driver can take ownership of the CSI-2 link and camera > > > sensor using interfaces exported via v4l2 sub-device. > > > > > > Switching ownership requires an interface with two different hardware > > > modules inside IVSC. The software interface to these modules is via > > > Intel MEI (The Intel Management Engine) commands. These two hardware > > > modules have two different MEI UUIDs to enumerate. These hardware > > modules are: > > > - ACE (Algorithm Context Engine): This module is for algorithm > > > computing when IVSC owns camera sensor. Also ACE module controls > > > camera sensor's ownership. This hardware module is used to set ownership of > > camera sensor. > > > - CSI (Camera Serial Interface): This module is used to route camera > > > sensor data either to IVSC or to host for IPU driver and application. > > > > > > IVSC also provides a privacy mode. When privacy mode is turned on, > > > camera sensor can't be used. This means that both ACE and host IPU > > > can't get image data. And when this mode is turned on, users are > > > informed via > > > v4l2 control API. > > > > > > In summary, to acquire ownership of camera by IPU driver, first ACE > > > module needs to be informed of ownership and then to setup MIPI CSI-2 > > > link for the camera sensor and IPU. > > > > > > Implementation: > > > There are two different drivers to handle ACE and CSI hardware modules > > > inside IVSC. > > > - ivsc_csi: MEI client driver to send commands and receive > > > notifications from CSI module. > > > - ivsc_ace: MEI client driver to send commands and get status from > > > ACE module. > > > Interface is exposed via v4l2 sub-devcie APIs to acquire and release > > > camera sensor and CSI-2 link. > > > > Thanks for the update. > > > > Could you elaborate the decision of keeping the csi_bridge entirely separate > > from the cio2_bridge (to be turned to ipu_bridge first)? Both are doing > > essentially the same and using the same data structures, aren't they? > > yes, they're doing same thing to bridge the software nodes needed by > v4l2, but they have different type devices(pci and mei_client device) and > dependency. And they have same SSDB definition in DSDT, so the structures > are almost same. If there are differences, what are they? What comes to cio2_bridge, the fact that it's related to a PCI device doesn't seem to matter after initialisation so it could as well work with struct device. > > I have no idea what the ipu bridge would be like, but IVSC csi bridge can > be configured via kconfig to enable/disable. Please work out the details with Bingbu. And please do wrap your lines at 74 or so. -- Kind regards, Sakari Ailus
