Hi Sakari, > -----Original Message----- > From: Sakari Ailus <sakari.ailus@xxxxxxxxxxxxxxx> > Sent: Friday, February 17, 2023 6:50 PM > > Hi Wentong, > > On Fri, Feb 17, 2023 at 06:28:32AM +0000, Wu, Wentong wrote: > > > > > > > -----Original Message----- > > > From: Bingbu Cao <bingbu.cao@xxxxxxxxxxxxxxx> > > > Sent: Friday, February 17, 2023 9:44 AM > > > > > > Hi, Sakari, > > > > > > On 2/16/23 9:12 PM, Sakari Ailus wrote: > > > > Hi Bingbu, Wentong, > > > > > > > > On Wed, Feb 15, 2023 at 08:09:50PM +0800, Bingbu Cao wrote: > > > >> > > > >> Hi, Wentong, > > > >> > > > >> On 2/15/23 5:43 PM, Laurent Pinchart wrote: > > > >>> Hello Wentong, > > > >>> > > > >>> On Mon, Feb 13, 2023 at 10:23:44AM +0800, Wentong Wu wrote: > > > >>>> Intel Visual Sensing Controller (IVSC), codenamed "Clover > > > >>>> Falls", is a companion chip designed to provide secure and low > > > >>>> power vision capability to IA platforms. IVSC is available in > > > >>>> existing commercial platforms from multiple OEMs. > > > >>>> > > > >>>> The primary use case of IVSC is to bring in context awareness. > > > >>>> IVSC interfaces directly with the platform main camera sensor > > > >>>> via a > > > >>>> CSI-2 link and processes the image data with the embedded AI > > > >>>> engine. The detected events are sent over I2C to ISH (Intel > > > >>>> Sensor > > > >>>> Hub) for additional data fusion from multiple sensors. The > > > >>>> fusion results are used to implement advanced use cases like: > > > >>>> - Face detection to unlock screen > > > >>>> - Detect user presence to manage backlight setting or waking > > > >>>> up system > > > >>> > > > >>> Do you have plan to support these features in the ivsc driver in > > > >>> the future ? > > > >>> > > > >>>> Since the Image Processing Unit(IPU) used on the host processor > > > >>>> needs to configure the CSI-2 link in normal camera usages, the > > > >>>> CSI-2 link and camera sensor can only be used in > > > >>>> mutually-exclusive ways by host IPU and IVSC. By default the > > > >>>> IVSC owns the CSI-2 link and camera sensor. The IPU driver can > > > >>>> take ownership of the CSI-2 link and camera sensor using interfaces > provided by this IVSC driver. > > > >>>> > > > >>>> Switching ownership requires an interface with two different > > > >>>> hardware modules inside IVSC. The software interface to these > > > >>>> modules is via Intel MEI (The Intel Management Engine) commands. > > > >>>> These two hardware modules have two different MEI UUIDs to > enumerate. > > > These hardware modules are: > > > >>>> - ACE (Algorithm Context Engine): This module is for algorithm > > > >>>> computing when IVSC owns camera sensor. Also ACE module > > > >>>> controls camera sensor's ownership. This hardware module is > > > >>>> used to set ownership > > > of camera sensor. > > > >>>> - CSI (Camera Serial Interface): This module is used to route > > > >>>> camera sensor data either to IVSC or to host for IPU driver and > application. > > > >>>> > > > >>>> IVSC also provides a privacy mode. When privacy mode is turned > > > >>>> on, camera sensor can't be used. This means that both ACE and > > > >>>> host IPU can't get image data. And when this mode is turned on, > > > >>>> host IPU driver is informed via a registered callback, so that user can be > notified. > > > >>> > > > >>> How does the privacy mode work, and how can the user trust that > > > >>> the closed-source IVSC and IME firmwares will honour the privacy > settings ? > > > > > > As I know, without IVSC, once user enable the privacy mode, the > > > Intel Converged Security Engine will configure the IPU camera mask > > > (security register), which will mask the specific CSI2 port and > > > produce dummy imaging data. For the case with IVSC, there is no final > solution on Linux so far I think. > > > > > > Wentong, is IVSC trying to cut off the stream and then notify user and IPU? > > > > yes > > Does the CSI-2 transmitter on IVCS go to some LP mode during this time, Yes, The low power mode is following the D-Phy spec. > or does > the receiver need to initialise the bus again when the stream resuems? No, it's the same link. > > > > > > > > > >>> > > > >> > > > >> Continue with question from Laurent, > > > >> > > > >> How IVSC handle the privacy request from user? Is there some > > > >> notification mechanism to user-space? > > > > IVSC has already defined privacy callback for host IPU/camera driver. > > > > > > I'd have concern if IVSC driver > > > >> need private callback to request back-end(e.g. ISP driver) to > > > >> handle stream > > > cutting. > > > > > > > > How does the privacy mode work, does it just pass zeroes (or other > > > > dummy > > > > data) towards the host or nothing? > > > > No data on CSI transmitter side > > Can it stop in the middle of the frame? No, Or is it guaranteed to produce full frames > (assuming the sensor does)? Yes, full frame will be guaranteed as camera sensor does. > > > > > > > > > > > A V4L2 control can be used for the purpose of passing the > > > > information to the user space at least. > > > > I will take some time to review V4L2 sub-device and control mechanism, > > and then update the driver. > > -- > Kind regards, > > Sakari Ailus
