Hello, This is the architecture overview we put together for Qualcomm MSM camera support in linux-media tree. Your comments are very much appreciated! Introduction ============ This is the video4linux driver for Qualcomm MSM (Mobile Station Modem) camera. The driver supports video and camera functionality on Qualcomm MSM SoC. It supports camera sensors provided by OEMs with parallel/MIPI interfaces, and operates in continuous streaming, snapshot, or video recording modes. Hardware description ==================== MSM camera hardware contains the following components: 1. One or more camera sensors controlled via I2C bus, and data outputs on AXI or MIPI bus. 2. Video Front End (VFE) core is an image signal processing hardware pipeline that takes the sensor output, does the necessary processing, and outputs to a buffer for V4L2 application. VFE is clocked by PCLK (pixel clock) from the sensor. Software description ==================== The driver encapsulates the low-level hardware management and aligns the interface to V4L2 driver framework. There is also a user space camera service daemon which handles events from driver and changes settings accordingly. During boot-up, the sensor is probed for and if the probe succeeds /dev/video0 and /dev/msm_camera/config0 device nodes are created. The /dev/video0 node is used for video buffer allocation in the kernel and for receiving V4L2 ioctls for controlling the camera hardware (VFE, sensors). The /dev/msm_camera/config0 node is used for sending commands and other statistics available from the hardware to the camera service daemon. Note that if more than one camera sensor is detected, there will be /dev/video1 and /dev/msm_camera/config1 device nodes created as well. Design ====== For MSM camera IC, significant portion of image processing and optimization codes are proprietary, so they cannot sit in kernel space. This plays an important role when making design decisions. Our design is to have a light-weighted kernel driver, and put the proprietary code in a user space daemon. The daemon polls on events from /dev/msm_camera/config0 device in the form of v4l2_event. The events could either be asynchronous (generated by the hardware), or synchronous (control command from the application). Based on the events it receives, the daemon will send appropriate control commands to the hardware. +-------------+ +----------------+ | Application | | Service Daemon | +-------------+ +----------------+ User Space .......................................................... +--------------------------------------+ Kernel Space | V4L2 | +--------------------------------------+ +---------+ +--------+ +-------+ | VFE/ISP | | Sensor | | Flash | +---------+ +--------+ +-------+ Power Management ================ None at this point. SMP/multi-core ============== Mutex is used for synchronization of threads accessing the driver simultaneously. Between hardware interrupt handler and threads, we use spinlock to make sure locking is done properly. Security ======== None. Interface ========= The driver uses V4L2 API for user kernel interaction. Refer to http://v4l2spec.bytesex.org/spec-single/v4l2.html. Between camera service daemon and the driver, we have customized IOCTL commands associated with /dev/msm_camera/config0 node, that controls MSM camera hardware. The list of IOCTLs are (declarations can be found in include/media/msm_camera.h): MSM_CAM_IOCTL_GET_SENSOR_INFO Get the basic information such as name, flash support for the detected sensor. MSM_CAM_IOCTL_SENSOR_IO_CFG Get or set sensor configurations: fps, line_pf, pixels_pl, exposure and gain, etc. The setting is stored in sensor_cfg_data structure. MSM_CAM_IOCTL_CONFIG_VFE Change settings of different components of VFE hardware. MSM_CAM_IOCTL_CTRL_CMD_DONE Notify the driver that the ctrl command is finished. MSM_CAM_IOCTL_RELEASE_STATS_BUFFER Notify the driver that the service daemon has done processing the stats buffer, and is returning it to the driver. MSM_CAM_IOCTL_AXI_CONFIG Configure AXI bus parameters (frame buffer addresses, offsets) to the VFE hardware. Driver parameters ================= None. Config options ============== MSM_CAMERA in drivers/media/video/msm/Kconfig file Dependencies ============ PMIC, I2C, Clock, GPIO User space utilities ==================== A daemon process in the user space called mm-qcamera-daemon is polling on events from the driver. This daemon is required in order for the V4L2 client application to run, and it's started by the init script. Other ===== To do ===== 1. Eliminate creation of /dev/msm_camera/config0 by routing private ioctls through /dev/video0. 2. Create sub devices for sensor and VFE. -- Sent by an employee of the Qualcomm Innovation Center, Inc. The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum. -- -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
