Hi Conor, Thank you for the review. On Thu, Nov 24, 2022 at 9:31 PM Conor Dooley <conor@xxxxxxxxxx> wrote: > > On Thu, Nov 24, 2022 at 05:22:07PM +0000, Prabhakar wrote: > > From: Lad Prabhakar <prabhakar.mahadev-lad.rj@xxxxxxxxxxxxxx> > > > > On the AX45MP core, cache coherency is a specification option so it may > > How about: > "Cache coherency is an option feature of the AX45MP core, so it may not > be supported." > Sure, I'll update that. > I keep finding that sentence kinda hard.. > > > In this case DMA will fail. > "The AX45MP predates the standard extensions for cache management, so an > alternate approach is required to support non-coherent DMA for SoCs > where this feature is not available, such as the Renesas RZ/Five." > > (you've gotta explain somewhere why this is in drivers/soc/renesas lol) > I dont want to touch that fiery topic ;) > > As a workaround, firstly we > > How about: > " Since the cache management instructions cannot be used, we instead > allocate..." > Sure, I'll update that. > > allocate a global dma coherent pool from which DMA allocations are taken > > and marked as non-cacheable + bufferable using the PMA region as specified > > in the device tree. Synchronization callbacks are implemented to > > synchronize when doing DMA transactions. > > > > The Andes AX45MP core has a Programmable Physical Memory Attributes (PMA) > > block that allows dynamic adjustment of memory attributes in the runtime. > > It contains a configurable amount of PMA entries implemented as CSR > > registers to control the attributes of memory locations in interest. > > > > Below are the memory attributes supported: > > * Device, Non-bufferable > > * Device, bufferable > > * Memory, Non-cacheable, Non-bufferable > > * Memory, Non-cacheable, Bufferable > > * Memory, Write-back, No-allocate > > * Memory, Write-back, Read-allocate > > * Memory, Write-back, Write-allocate > > * Memory, Write-back, Read and Write-allocate > > > > This patch adds support to configure the memory attributes of the memory > > regions as passed from the l2 cache node and exposes the cache management > > ops. > > > > More info about PMA (section 10.3): > > Link: http://www.andestech.com/wp-content/uploads/AX45MP-1C-Rev.-5.0.0-Datasheet.pdf > > But yeah, this is basically the sort of stuff that'd be nice to have in > the previous patch! > Agreed, I'll include this in the binding patch too. > > Signed-off-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@xxxxxxxxxxxxxx> > > --- > > RFC v3 -> v4 > > * Made use of runtime patching instead of compile time > > * Now just exposing single function ax45mp_no_iocp_cmo() for CMO handling > > * Added a check to make sure cache line size is always 64 bytes > > * Renamed folder rzf -> rzfive > > * Improved Kconfig description > > * Dropped L2 cache configuration > > * Dropped unnecessary casts > > * Fixed comments pointed by Geert, apart from use of PTR_ALIGN_XYZ() macros. > > --- > > arch/riscv/include/asm/cacheflush.h | 8 + > > arch/riscv/include/asm/errata_list.h | 32 +- > > drivers/soc/renesas/Kconfig | 7 + > > drivers/soc/renesas/Makefile | 2 + > > drivers/soc/renesas/rzfive/Kconfig | 6 + > > drivers/soc/renesas/rzfive/Makefile | 3 + > > drivers/soc/renesas/rzfive/ax45mp_cache.c | 415 ++++++++++++++++++++++ > > drivers/soc/renesas/rzfive/ax45mp_sbi.h | 29 ++ > > 8 files changed, 496 insertions(+), 6 deletions(-) > > create mode 100644 drivers/soc/renesas/rzfive/Kconfig > > create mode 100644 drivers/soc/renesas/rzfive/Makefile > > create mode 100644 drivers/soc/renesas/rzfive/ax45mp_cache.c > > create mode 100644 drivers/soc/renesas/rzfive/ax45mp_sbi.h > > > diff --git a/drivers/soc/renesas/Kconfig b/drivers/soc/renesas/Kconfig > > index 660498252ec5..e7810256c60d 100644 > > --- a/drivers/soc/renesas/Kconfig > > +++ b/drivers/soc/renesas/Kconfig > > @@ -340,9 +340,16 @@ if RISCV > > config ARCH_R9A07G043 > > bool "RISC-V Platform support for RZ/Five" > > select ARCH_RZG2L > > + select AX45MP_L2_CACHE > > + select DMA_GLOBAL_POOL > > + select ERRATA_ANDES > > + select ERRATA_ANDES_CMO > > + select RISCV_DMA_NONCOHERENT > > Is this not redundant due to the select by ERRATA_ANDES_CMO? > Indeed, I'll drop it. > > help > > This enables support for the Renesas RZ/Five SoC. > > > > +source "drivers/soc/renesas/rzfive/Kconfig" > > + > > endif # RISCV > > > > config RST_RCAR > > > diff --git a/drivers/soc/renesas/rzfive/Makefile b/drivers/soc/renesas/rzfive/Makefile > > new file mode 100644 > > index 000000000000..2012e7fb978d > > --- /dev/null > > +++ b/drivers/soc/renesas/rzfive/Makefile > > @@ -0,0 +1,3 @@ > > +# SPDX-License-Identifier: GPL-2.0 > > + > > +obj-$(CONFIG_AX45MP_L2_CACHE) += ax45mp_cache.o > > diff --git a/drivers/soc/renesas/rzfive/ax45mp_cache.c b/drivers/soc/renesas/rzfive/ax45mp_cache.c > > new file mode 100644 > > index 000000000000..4e0d0545d3af > > Mainly just whizzing through the driver itself.. > > > +static int ax45mp_configure_pma_regions(struct device_node *np) > > +{ > > + const char *propname = "andestech,pma-regions"; > > + u32 start, size, flags; > > + unsigned int entry_id; > > + unsigned int i; > > + int count; > > + int ret; > > + > > + count = of_property_count_elems_of_size(np, propname, sizeof(u32) * 3); > > + if (count < 0) > > + return count; > > + > > + if (count > AX45MP_MAX_PMA_REGIONS) > > + return -EINVAL; > > + > > + for (i = 0, entry_id = 0 ; entry_id < count ; i += 3, entry_id++) { > > + of_property_read_u32_index(np, propname, i, &start); > > + of_property_read_u32_index(np, propname, i + 1, &size); > > + of_property_read_u32_index(np, propname, i + 2, &flags); > > + ret = ax45mp_sbi_set_pma(start, size, flags, entry_id); > > + if (!ret) > > + pr_err("Failed to setup PMA region 0x%x - 0x%x flags: 0x%x", > > + start, start + size, flags); > > I have to ask - is it okay to just continue here if a RMA region setup > fails? > Ok so incase of failures "ebreak" is called, so continuing doesn't cause any harm I guess. > > + } > > + > > + return 0; > > +} > > + > > > +static bool ax45mp_cpu_cache_controlable(void) > > +{ > > + return (((ax45mp_cpu_get_micm_cfg_status() & AX45MP_MICM_CFG_ISZ_MASK) || > > + (ax45mp_cpu_get_mdcm_cfg_status() & AX45MP_MDCM_CFG_DSZ_MASK)) && > > + (ax45mp_cpu_get_misa_cfg_status() & AX45MP_MISA_20_MASK) && > > + (ax45mp_cpu_get_mmsc_cfg_status() & AX45MP_MMSC_CFG_CCTLCSR_MASK) && > > + (ax45mp_cpu_get_mcache_ctl_status() & AX45MP_MCACHE_CTL_CCTL_SUEN_MASK)); > > That's a bit of a mouthful lol! > > > > +static void ax45mp_cpu_dma_inval_range(void *vaddr, size_t size) > > Not mine to look after so /shrug but this looks like the sort of thing > that could do with a comment or two explaining the invalidation process. > Sure, I'll add some comments. > > +{ > > + char cache_buf[2][AX45MP_MAX_CACHE_LINE_SIZE]; > > + unsigned long start = (unsigned long)vaddr; > > + unsigned long end = start + size; > > + unsigned long old_start = start; > > + unsigned long old_end = end; > > + unsigned long line_size; > > + unsigned long flags; > > + > > + if (static_branch_unlikely(&ax45mp_l2c_configured) && !ax45mp_priv) > > + return; > > + > > + if (unlikely(start == end)) > > + return; > > + > > + line_size = ax45mp_priv->ax45mp_cache_line_size; > > + > > + memset(&cache_buf, 0x0, sizeof(cache_buf)); > > + start = start & (~(line_size - 1)); > > + end = ((end + line_size - 1) & (~(line_size - 1))); > > + > > + local_irq_save(flags); > > + if (unlikely(start != old_start)) > > + memcpy(&cache_buf[0][0], (void *)start, line_size); > > + > > + if (unlikely(end != old_end)) > > + memcpy(&cache_buf[1][0], (void *)(old_end & (~(line_size - 1))), line_size); > > + > > + ax45mp_cpu_dcache_inval_range(vaddr, (void *)end, line_size); > > + > > + if (unlikely(start != old_start)) > > + memcpy((void *)start, &cache_buf[0][0], (old_start & (line_size - 1))); > > + > > + if (unlikely(end != old_end)) > > + memcpy((void *)(old_end + 1), > > + &cache_buf[1][(old_end & (line_size - 1)) + 1], > > + end - old_end - 1); > > + > > + local_irq_restore(flags); > > +} > > > +static int ax45mp_configure_l2_cache(struct device_node *np) > > +{ > > + int ret; > > + > > + ret = of_property_read_u32(np, "cache-line-size", &ax45mp_priv->ax45mp_cache_line_size); > > + if (ret) { > > + pr_err("Failed to get cache-line-size defaulting to 64 bytes\n"); > ^ > Looks like you need a comma here... > OK. > > + ax45mp_priv->ax45mp_cache_line_size = SZ_64; > > + } > > + > > + if (ax45mp_priv->ax45mp_cache_line_size != SZ_64) { > > + pr_err("Expected cache-line-size to 64 bytes (found:%u). Defaulting to 64 bytes\n", > ^ > ...and a "be" here. > > Would you also benefit from a pr_fmt here since you have no device? Or > else you could save the dev to your ax45mp_priv and avail of dev_err > here? > Sure, I'll switch to dev_err() > > + ax45mp_priv->ax45mp_cache_line_size); > > + ax45mp_priv->ax45mp_cache_line_size = SZ_64; > > + } > > + > > + ax45mp_priv->ucctl_ok = ax45mp_cpu_cache_controlable(); > ^ > That function name is a typo, should be called ax45mp_cpu_cache_cache_controllable(). > Ok, I'll rename it to ax45mp_cpu_cache_controllable(). > > + ax45mp_priv->l2cache_enabled = ax45mp_cpu_l2c_ctl_status() & AX45MP_L2_CACHE_CTL_CEN_MASK; > > + > > + return 0; > > +} > > + > > +static int ax45mp_l2c_probe(struct platform_device *pdev) > > +{ > > + struct device_node *np = pdev->dev.of_node; > > + int ret; > > + > > + ax45mp_priv = devm_kzalloc(&pdev->dev, sizeof(*ax45mp_priv), GFP_KERNEL); > > + if (!ax45mp_priv) > > + return -ENOMEM; > > + > > + ax45mp_priv->l2c_base = devm_of_iomap(&pdev->dev, pdev->dev.of_node, 0, NULL); > > + if (!ax45mp_priv->l2c_base) { > > + ret = -ENOMEM; > > + goto l2c_err; > > + } > > + > > + ret = ax45mp_configure_l2_cache(np); > > + if (ret) > > + goto l2c_err; > > + > > + ret = ax45mp_configure_pma_regions(np); > > + if (ret) > > + goto l2c_err; > > + > > + static_branch_disable(&ax45mp_l2c_configured); > > + > > + return 0; > > + > > +l2c_err: > > + devm_kfree(&pdev->dev, ax45mp_priv); > > + ax45mp_priv = NULL; > > + return ret; > > +} > > > diff --git a/drivers/soc/renesas/rzfive/ax45mp_sbi.h b/drivers/soc/renesas/rzfive/ax45mp_sbi.h > > new file mode 100644 > > index 000000000000..1604874954d0 > > --- /dev/null > > +++ b/drivers/soc/renesas/rzfive/ax45mp_sbi.h > > @@ -0,0 +1,29 @@ > > +/* SPDX-License-Identifier: GPL-2.0+ */ > > + > > +#ifndef __AX45MP_SBI_H > > +#define __AX45MP_SBI_H > > + > > +#define SBI_EXT_ANDES 0x0900031E > > + > > +enum ax45mp_sbi_ext_fid { > > + AX45MP_SBI_EXT_GET_MCACHE_CTL_STATUS = 0, > > Is that zero not implied? > I had a feeling we always had to explicitly specify in the Linux coding standard. Cheers, Prabhakar
