On Thu, Sep 14, 2023 at 6:17 PM Sakari Ailus <sakari.ailus@xxxxxx> wrote: > > Hi Shenjiu, > > Thanks for the update. > > On Thu, Sep 14, 2023 at 01:54:02PM +0800, Shengjiu Wang wrote: > > Audio signal processing has the requirement for memory to > > memory similar as Video. > > > > This patch is to add this support in v4l2 framework, defined > > new buffer type V4L2_BUF_TYPE_AUDIO_CAPTURE and > > V4L2_BUF_TYPE_AUDIO_OUTPUT, defined new format v4l2_audio_format > > for audio case usage. > > > > Defined V4L2_AUDIO_FMT_LPCM format type for audio. > > This would be nicer as a separate patch. Also see the related comments > below. OK, will separate it. > > > > > Defined V4L2_CAP_AUDIO_M2M capability type for audio memory > > to memory case. > > > > The created audio device is named "/dev/v4l-audioX". > > > > Signed-off-by: Shengjiu Wang <shengjiu.wang@xxxxxxx> > > --- > > .../userspace-api/media/v4l/audio-formats.rst | 15 +++++ > > .../userspace-api/media/v4l/buffer.rst | 6 ++ > > .../userspace-api/media/v4l/dev-audio.rst | 63 +++++++++++++++++++ > > .../userspace-api/media/v4l/devices.rst | 1 + > > .../media/v4l/pixfmt-aud-lpcm.rst | 31 +++++++++ > > .../userspace-api/media/v4l/pixfmt.rst | 1 + > > .../media/v4l/vidioc-enum-fmt.rst | 2 + > > .../userspace-api/media/v4l/vidioc-g-fmt.rst | 4 ++ > > .../media/v4l/vidioc-querycap.rst | 3 + > > .../media/videodev2.h.rst.exceptions | 2 + > > .../media/common/videobuf2/videobuf2-v4l2.c | 4 ++ > > drivers/media/v4l2-core/v4l2-dev.c | 17 +++++ > > drivers/media/v4l2-core/v4l2-ioctl.c | 53 ++++++++++++++++ > > include/media/v4l2-dev.h | 2 + > > include/media/v4l2-ioctl.h | 34 ++++++++++ > > include/uapi/linux/videodev2.h | 25 ++++++++ > > 16 files changed, 263 insertions(+) > > create mode 100644 Documentation/userspace-api/media/v4l/audio-formats.rst > > create mode 100644 Documentation/userspace-api/media/v4l/dev-audio.rst > > create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-aud-lpcm.rst > > > > diff --git a/Documentation/userspace-api/media/v4l/audio-formats.rst b/Documentation/userspace-api/media/v4l/audio-formats.rst > > new file mode 100644 > > index 000000000000..bc52712d20d3 > > --- /dev/null > > +++ b/Documentation/userspace-api/media/v4l/audio-formats.rst > > @@ -0,0 +1,15 @@ > > +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later > > + > > +.. _audio-formats: > > + > > +************* > > +Audio Formats > > +************* > > + > > +These formats are used for :ref:`audio` interface only. > > + > > + > > +.. toctree:: > > + :maxdepth: 1 > > + > > + pixfmt-aud-lpcm > > diff --git a/Documentation/userspace-api/media/v4l/buffer.rst b/Documentation/userspace-api/media/v4l/buffer.rst > > index 04dec3e570ed..80cf2cb20dfe 100644 > > --- a/Documentation/userspace-api/media/v4l/buffer.rst > > +++ b/Documentation/userspace-api/media/v4l/buffer.rst > > @@ -438,6 +438,12 @@ enum v4l2_buf_type > > * - ``V4L2_BUF_TYPE_META_OUTPUT`` > > - 14 > > - Buffer for metadata output, see :ref:`metadata`. > > + * - ``V4L2_BUF_TYPE_AUDIO_CAPTURE`` > > + - 15 > > + - Buffer for audio capture, see :ref:`audio`. > > + * - ``V4L2_BUF_TYPE_AUDIO_OUTPUT`` > > + - 16 > > + - Buffer for audio output, see :ref:`audio`. > > > > > > .. _buffer-flags: > > diff --git a/Documentation/userspace-api/media/v4l/dev-audio.rst b/Documentation/userspace-api/media/v4l/dev-audio.rst > > new file mode 100644 > > index 000000000000..f9bcf0c7b056 > > --- /dev/null > > +++ b/Documentation/userspace-api/media/v4l/dev-audio.rst > > @@ -0,0 +1,63 @@ > > +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later > > + > > +.. _audiodev: > > + > > +****************** > > +audio Interface > > Capital "A"? OK, will modify it. > > > +****************** > > Too many asterisks (same a few lines above, too). ok, will update it. > > > + > > +The audio interface is implemented on audio device nodes. The audio device > > +which uses application software for modulation or demodulation. This > > +interface is intended for controlling and data streaming of such devices > > + > > +Audio devices are accessed through character device special files named > > +``/dev/v4l-audio`` > > + > > +Querying Capabilities > > +===================== > > + > > +Device nodes supporting the audio capture and output interface set the > > +``V4L2_CAP_AUDIO_M2M`` flag in the ``device_caps`` field of the > > +:c:type:`v4l2_capability` structure returned by the :c:func:`VIDIOC_QUERYCAP` > > +ioctl. > > + > > +At least one of the read/write or streaming I/O methods must be supported. > > + > > + > > +Data Format Negotiation > > +======================= > > + > > +The audio device uses the :ref:`format` ioctls to select the capture format. > > +The audio buffer content format is bound to that selected format. In addition > > +to the basic :ref:`format` ioctls, the :c:func:`VIDIOC_ENUM_FMT` ioctl must be > > +supported as well. > > + > > +To use the :ref:`format` ioctls applications set the ``type`` field of the > > +:c:type:`v4l2_format` structure to ``V4L2_BUF_TYPE_AUDIO_CAPTURE`` or to > > +``V4L2_BUF_TYPE_AUDIO_OUTPUT``. Both drivers and applications must set the > > +remainder of the :c:type:`v4l2_format` structure to 0. > > + > > +.. c:type:: v4l2_audio_format > > + > > +.. tabularcolumns:: |p{1.4cm}|p{2.4cm}|p{13.5cm}| > > + > > +.. flat-table:: struct v4l2_audio_format > > + :header-rows: 0 > > + :stub-columns: 0 > > + :widths: 1 1 2 > > + > > + * - __u32 > > + - ``rate`` > > + - The sample rate, set by the application. The range is [5512, 768000]. > > + * - __u32 > > + - ``format`` > > + - The sample format, set by the application. format is defined as > > + SNDRV_PCM_FORMAT_S8, SNDRV_PCM_FORMAT_U8, ..., > > + * - __u32 > > + - ``channels`` > > + - The channel number, set by the application. channel number range is > > + [1, 32]. > > + * - __u32 > > + - ``buffersize`` > > + - Maximum buffer size in bytes required for data. The value is set by the > > + driver. > > diff --git a/Documentation/userspace-api/media/v4l/devices.rst b/Documentation/userspace-api/media/v4l/devices.rst > > index 8bfbad65a9d4..8261f3468489 100644 > > --- a/Documentation/userspace-api/media/v4l/devices.rst > > +++ b/Documentation/userspace-api/media/v4l/devices.rst > > @@ -24,3 +24,4 @@ Interfaces > > dev-event > > dev-subdev > > dev-meta > > + dev-audio > > diff --git a/Documentation/userspace-api/media/v4l/pixfmt-aud-lpcm.rst b/Documentation/userspace-api/media/v4l/pixfmt-aud-lpcm.rst > > new file mode 100644 > > index 000000000000..f9ebe2a05f69 > > --- /dev/null > > +++ b/Documentation/userspace-api/media/v4l/pixfmt-aud-lpcm.rst > > @@ -0,0 +1,31 @@ > > +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later > > + > > +.. _v4l2-aud-fmt-lpcm: > > + > > +************************* > > +V4L2_AUDIO_FMT_LPCM ('LPCM') > > +************************* > > + > > +Linear Pulse-Code Modulation (LPCM) > > + > > + > > +Description > > +=========== > > + > > +This describes audio format used by the audio memory to memory driver. > > + > > +It contains the following fields: > > + > > +.. flat-table:: > > + :widths: 1 4 > > + :header-rows: 1 > > + :stub-columns: 0 > > + > > + * - Field > > + - Description > > + * - u32 samplerate; > > + - which is the number of times per second that samples are taken. > > + * - u32 sampleformat; > > + - which determines the number of possible digital values that can be used to represent each sample > > 80 characters (or less) per line, please. Ok, will change it. > > Which values could this field have and what do they signify? The values are SNDRV_PCM_FORMAT_S8, SNDRV_PCM_FORMAT_U8... which are the PCM format, defined in ALSA. > > > + * - u32 channels; > > + - channel number for each sample. > > I suppose the rest of the buffer would be samples? This should be > documented. I think there are also different ways the data could be > arrangeed and this needs to be documented, too. All data in the buffer are the samples, the 'samplerate', 'sampleformat' 'channels' I list here is try to describe the samples. I was confused how to write this document, so I list the characters. Best regards Wang Shengjiu