On 19/03/2024 4:14 am, Shengjiu Wang wrote: > On Tue, Mar 19, 2024 at 10:24 AM Shengjiu Wang <shengjiu.wang@xxxxxxxxx> wrote: >> >> On Mon, Mar 18, 2024 at 11:56 PM Hans Verkuil <hverkuil@xxxxxxxxx> wrote: >>> >>> Hi Shengjiu, >>> >>> I was about to prepare a pull request for this when I realized that >>> entries for the new drivers need to be added to the MAINTAINERS file. >>> >>> Can you post patches 'v14 17/16' and 'v14/18/16' adding entries for each >>> new driver? >> >> Ok, How about the change below? >> >> VIRTUAL MEM2MEM DRIVER FOR AUDIO >> M: Hans Verkuil <hverkuil@xxxxxxxxx> >> M: Shengjiu Wang <shengjiu.wang@xxxxxxxxx> >> L: linux-media@xxxxxxxxxxxxxxx >> S: Maintained >> W: https://linuxtv.org >> T: git git://linuxtv.org/media_tree.git >> F: drivers/media/test-drivers/vim2m-audio.c >> >> NXP ASRC V4L2 MEM2MEM DRIVERS >> M: Shengjiu Wang <shengjiu.wang@xxxxxxxxx> >> L: linux-media@xxxxxxxxxxxxxxx >> S: Maintained >> W: https://linuxtv.org >> T: git git://linuxtv.org/media_tree.git >> F: drivers/media/platform/nxp/imx-asrc.c >> > > BTW, should I merge this change to PATCH 15/16 and PATCH 16/16? > That I will resend the whole patch series. Seems this is a better way. That's fine! Regards, Hans > > Best regrads > Shengjiu Wang > >> best regards >> wang shengjiu >> >>> >>> Thank you, >>> >>> Hans >>> >>> On 11/03/2024 10:53 am, Shengjiu Wang wrote: >>>> Audio signal processing also has the requirement for memory to >>>> memory similar as Video. >>>> >>>> This asrc memory to memory (memory ->asrc->memory) case is a non >>>> real time use case. >>>> >>>> User fills the input buffer to the asrc module, after conversion, then asrc >>>> sends back the output buffer to user. So it is not a traditional ALSA playback >>>> and capture case. >>>> >>>> It is a specific use case, there is no reference in current kernel. >>>> v4l2 memory to memory is the closed implementation, v4l2 current >>>> support video, image, radio, tuner, touch devices, so it is not >>>> complicated to add support for this specific audio case. >>>> >>>> Because we had implemented the "memory -> asrc ->i2s device-> codec" >>>> use case in ALSA. Now the "memory->asrc->memory" needs >>>> to reuse the code in asrc driver, so the first 3 patches is for refining >>>> the code to make it can be shared by the "memory->asrc->memory" >>>> driver. >>>> >>>> The main change is in the v4l2 side, A /dev/vl4-audioX will be created, >>>> user applications only use the ioctl of v4l2 framework. >>>> >>>> Other change is to add memory to memory support for two kinds of i.MX ASRC >>>> module. >>>> >>>> changes in v14: >>>> - document the reservation of 'AUXX' fourcc format. >>>> - add v4l2_audfmt_to_fourcc() definition. >>>> >>>> changes in v13 >>>> - change 'pixelformat' to 'audioformat' in dev-audio-mem2mem.rst >>>> - add more description for clock drift in ext-ctrls-audio-m2m.rst >>>> - Add "media: v4l2-ctrls: add support for fraction_bits" from Hans >>>> to avoid build issue for kernel test robot >>>> >>>> changes in v12 >>>> - minor changes according to comments >>>> - drop min_buffers_needed = 1 and V4L2_CTRL_FLAG_UPDATE flag >>>> - drop bus_info >>>> >>>> changes in v11 >>>> - add add-fixed-point-test-controls in vivid. >>>> - add v4l2_ctrl_fp_compose() helper function for min and max >>>> >>>> changes in v10 >>>> - remove FIXED_POINT type >>>> - change code base on media: v4l2-ctrls: add support for fraction_bits >>>> - fix issue reported by kernel test robot >>>> - remove module_alias >>>> >>>> changes in v9: >>>> - add MEDIA_ENT_F_PROC_AUDIO_RESAMPLER. >>>> - add MEDIA_INTF_T_V4L_AUDIO >>>> - add media controller support >>>> - refine the vim2m-audio to support 8k<->16k conversion. >>>> >>>> changes in v8: >>>> - refine V4L2_CAP_AUDIO_M2M to be 0x00000008 >>>> - update doc for FIXED_POINT >>>> - address comments for imx-asrc >>>> >>>> changes in v7: >>>> - add acked-by from Mark >>>> - separate commit for fixed point, m2m audio class, audio rate controls >>>> - use INTEGER_MENU for rate, FIXED_POINT for rate offset >>>> - remove used fmts >>>> - address other comments for Hans >>>> >>>> changes in v6: >>>> - use m2m_prepare/m2m_unprepare/m2m_start/m2m_stop to replace >>>> m2m_start_part_one/m2m_stop_part_one, m2m_start_part_two/m2m_stop_part_two. >>>> - change V4L2_CTRL_TYPE_ASRC_RATE to V4L2_CTRL_TYPE_FIXED_POINT >>>> - fix warning by kernel test rebot >>>> - remove some unused format V4L2_AUDIO_FMT_XX >>>> - Get SNDRV_PCM_FORMAT from V4L2_AUDIO_FMT in driver. >>>> - rename audm2m to viaudm2m. >>>> >>>> changes in v5: >>>> - remove V4L2_AUDIO_FMT_LPCM >>>> - define audio pixel format like V4L2_AUDIO_FMT_S8... >>>> - remove rate and format in struct v4l2_audio_format. >>>> - Add V4L2_CID_ASRC_SOURCE_RATE and V4L2_CID_ASRC_DEST_RATE controls >>>> - updata document accordingly. >>>> >>>> changes in v4: >>>> - update document style >>>> - separate V4L2_AUDIO_FMT_LPCM and V4L2_CAP_AUDIO_M2M in separate commit >>>> >>>> changes in v3: >>>> - Modify documents for adding audio m2m support >>>> - Add audio virtual m2m driver >>>> - Defined V4L2_AUDIO_FMT_LPCM format type for audio. >>>> - Defined V4L2_CAP_AUDIO_M2M capability type for audio m2m case. >>>> - with modification in v4l-utils, pass v4l2-compliance test. >>>> >>>> changes in v2: >>>> - decouple the implementation in v4l2 and ALSA >>>> - implement the memory to memory driver as a platfrom driver >>>> and move it to driver/media >>>> - move fsl_asrc_common.h to include/sound folder >>>> >>>> Hans Verkuil (1): >>>> media: v4l2-ctrls: add support for fraction_bits >>>> >>>> Shengjiu Wang (15): >>>> ASoC: fsl_asrc: define functions for memory to memory usage >>>> ASoC: fsl_easrc: define functions for memory to memory usage >>>> ASoC: fsl_asrc: move fsl_asrc_common.h to include/sound >>>> ASoC: fsl_asrc: register m2m platform device >>>> ASoC: fsl_easrc: register m2m platform device >>>> media: uapi: Add V4L2_CAP_AUDIO_M2M capability flag >>>> media: v4l2: Add audio capture and output support >>>> media: uapi: Define audio sample format fourcc type >>>> media: uapi: Add V4L2_CTRL_CLASS_M2M_AUDIO >>>> media: uapi: Add audio rate controls support >>>> media: uapi: Declare interface types for Audio >>>> media: uapi: Add an entity type for audio resampler >>>> media: vivid: add fixed point test controls >>>> media: imx-asrc: Add memory to memory driver >>>> media: vim2m-audio: add virtual driver for audio memory to memory >>>> >>>> .../media/mediactl/media-types.rst | 11 + >>>> .../userspace-api/media/v4l/buffer.rst | 6 + >>>> .../userspace-api/media/v4l/common.rst | 1 + >>>> .../media/v4l/dev-audio-mem2mem.rst | 71 + >>>> .../userspace-api/media/v4l/devices.rst | 1 + >>>> .../media/v4l/ext-ctrls-audio-m2m.rst | 59 + >>>> .../userspace-api/media/v4l/pixfmt-audio.rst | 100 ++ >>>> .../userspace-api/media/v4l/pixfmt.rst | 1 + >>>> .../media/v4l/vidioc-enum-fmt.rst | 2 + >>>> .../media/v4l/vidioc-g-ext-ctrls.rst | 4 + >>>> .../userspace-api/media/v4l/vidioc-g-fmt.rst | 4 + >>>> .../media/v4l/vidioc-querycap.rst | 3 + >>>> .../media/v4l/vidioc-queryctrl.rst | 11 +- >>>> .../media/videodev2.h.rst.exceptions | 3 + >>>> .../media/common/videobuf2/videobuf2-v4l2.c | 4 + >>>> drivers/media/platform/nxp/Kconfig | 13 + >>>> drivers/media/platform/nxp/Makefile | 1 + >>>> drivers/media/platform/nxp/imx-asrc.c | 1256 +++++++++++++++++ >>>> drivers/media/test-drivers/Kconfig | 10 + >>>> drivers/media/test-drivers/Makefile | 1 + >>>> drivers/media/test-drivers/vim2m-audio.c | 793 +++++++++++ >>>> drivers/media/test-drivers/vivid/vivid-core.h | 2 + >>>> .../media/test-drivers/vivid/vivid-ctrls.c | 26 + >>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c | 9 + >>>> drivers/media/v4l2-core/v4l2-ctrls-api.c | 1 + >>>> drivers/media/v4l2-core/v4l2-ctrls-core.c | 93 +- >>>> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 10 + >>>> drivers/media/v4l2-core/v4l2-dev.c | 21 + >>>> drivers/media/v4l2-core/v4l2-ioctl.c | 66 + >>>> drivers/media/v4l2-core/v4l2-mem2mem.c | 13 +- >>>> include/media/v4l2-ctrls.h | 13 +- >>>> include/media/v4l2-dev.h | 2 + >>>> include/media/v4l2-ioctl.h | 34 + >>>> .../fsl => include/sound}/fsl_asrc_common.h | 60 + >>>> include/uapi/linux/media.h | 2 + >>>> include/uapi/linux/v4l2-controls.h | 9 + >>>> include/uapi/linux/videodev2.h | 50 +- >>>> sound/soc/fsl/fsl_asrc.c | 144 ++ >>>> sound/soc/fsl/fsl_asrc.h | 4 +- >>>> sound/soc/fsl/fsl_asrc_dma.c | 2 +- >>>> sound/soc/fsl/fsl_easrc.c | 233 +++ >>>> sound/soc/fsl/fsl_easrc.h | 6 +- >>>> 42 files changed, 3128 insertions(+), 27 deletions(-) >>>> create mode 100644 Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst >>>> create mode 100644 Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst >>>> create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-audio.rst >>>> create mode 100644 drivers/media/platform/nxp/imx-asrc.c >>>> create mode 100644 drivers/media/test-drivers/vim2m-audio.c >>>> rename {sound/soc/fsl => include/sound}/fsl_asrc_common.h (60%) >>>> >>> >
