On Wed, Dec 09, 2020 at 09:45:32AM +0100, Arnaud POULIQUEN wrote:
>
>
> On 12/9/20 1:53 AM, Mathieu Poirier wrote:
> > On Tue, Dec 08, 2020 at 07:35:18PM +0100, Arnaud POULIQUEN wrote:
> >> Hi Mathieu,
> >>
> >>
> >> On 11/26/20 10:06 PM, Mathieu Poirier wrote:
> >>> Introduce function rproc_detach() to enable the remoteproc
> >>> core to release the resources associated with a remote processor
> >>> without stopping its operation.
> >>>
> >>> Signed-off-by: Mathieu Poirier <mathieu.poirier@xxxxxxxxxx>
> >>> Reviewed-by: Peng Fan <peng.fan@xxxxxxx>
> >>> ---
> >>> drivers/remoteproc/remoteproc_core.c | 65 +++++++++++++++++++++++++++-
> >>> include/linux/remoteproc.h | 1 +
> >>> 2 files changed, 65 insertions(+), 1 deletion(-)
> >>>
> >>> diff --git a/drivers/remoteproc/remoteproc_core.c b/drivers/remoteproc/remoteproc_core.c
> >>> index 928b3f975798..f5adf05762e9 100644
> >>> --- a/drivers/remoteproc/remoteproc_core.c
> >>> +++ b/drivers/remoteproc/remoteproc_core.c
> >>> @@ -1667,7 +1667,7 @@ static int rproc_stop(struct rproc *rproc, bool crashed)
> >>> /*
> >>> * __rproc_detach(): Does the opposite of rproc_attach()
> >>> */
> >>> -static int __maybe_unused __rproc_detach(struct rproc *rproc)
> >>> +static int __rproc_detach(struct rproc *rproc)
> >>> {
> >>> struct device *dev = &rproc->dev;
> >>> int ret;
> >>> @@ -1910,6 +1910,69 @@ void rproc_shutdown(struct rproc *rproc)
> >>> }
> >>> EXPORT_SYMBOL(rproc_shutdown);
> >>>
> >>> +/**
> >>> + * rproc_detach() - Detach the remote processor from the
> >>> + * remoteproc core
> >>> + *
> >>> + * @rproc: the remote processor
> >>> + *
> >>> + * Detach a remote processor (previously attached to with rproc_actuate()).
> >>> + *
> >>> + * In case @rproc is still being used by an additional user(s), then
> >>> + * this function will just decrement the power refcount and exit,
> >>> + * without disconnecting the device.
> >>> + *
> >>> + * Function rproc_detach() calls __rproc_detach() in order to let a remote
> >>> + * processor know that services provided by the application processor are
> >>> + * no longer available. From there it should be possible to remove the
> >>> + * platform driver and even power cycle the application processor (if the HW
> >>> + * supports it) without needing to switch off the remote processor.
> >>> + */
> >>> +int rproc_detach(struct rproc *rproc)
> >>> +{
> >>> + struct device *dev = &rproc->dev;
> >>> + int ret;
> >>> +
> >>> + ret = mutex_lock_interruptible(&rproc->lock);
> >>> + if (ret) {
> >>> + dev_err(dev, "can't lock rproc %s: %d\n", rproc->name, ret);
> >>> + return ret;
> >>> + }
> >>> +
> >>> + if (rproc->state != RPROC_RUNNING && rproc->state != RPROC_ATTACHED) {
> >>> + ret = -EPERM;
> >>> + goto out;
> >>> + }
> >>> +
> >>> + /* if the remote proc is still needed, bail out */
> >>> + if (!atomic_dec_and_test(&rproc->power)) {
> >>> + ret = -EBUSY;
> >>> + goto out;
> >>> + }
> >>> +
> >>> + ret = __rproc_detach(rproc);
> >>> + if (ret) {
> >>> + atomic_inc(&rproc->power);
> >>> + goto out;
> >>> + }
> >>> +
> >>> + /* clean up all acquired resources */
> >>> + rproc_resource_cleanup(rproc);
> >>
> >> I started to test the series, I found 2 problems testing in STM32P1 board.
> >>
> >> 1) the resource_table pointer is unmapped if the firmware has been booted by the
> >> Linux, generating a crash in rproc_free_vring.
> >> I attached a fix at the end of the mail.
> >>
> >
> > I have reproduced the condition on my side and confirm that your solution is
> > correct. See below for a minor comment.
> >
> >> 2) After the detach, the rproc state is "detached"
> >> but it is no longer possible to re-attach to it correctly.
> >> Neither if the firmware is standalone, nor if it has been booted
> >> by the Linux.
> >>
> >
> > Did you update your FW image? If so, I need to run the same one.
> >
> >> I did not investigate, but the issue is probably linked to the resource
> >> table address which is set to NULL.
> >>
> >> So we either have to fix the problem in order to attach or forbid the transition.
> >>
> >>
> >> Regards,
> >> Arnaud
> >>
> >>> +
> >>> + rproc_disable_iommu(rproc);
> >>> +
> >>> + /*
> >>> + * Set the remote processor's table pointer to NULL. Since mapping
> >>> + * of the resource table to a virtual address is done in the platform
> >>> + * driver, unmapping should also be done there.
> >>> + */
> >>> + rproc->table_ptr = NULL;
> >>> +out:
> >>> + mutex_unlock(&rproc->lock);
> >>> + return ret;
> >>> +}
> >>> +EXPORT_SYMBOL(rproc_detach);
> >>> +
> >>> /**
> >>> * rproc_get_by_phandle() - find a remote processor by phandle
> >>> * @phandle: phandle to the rproc
> >>> diff --git a/include/linux/remoteproc.h b/include/linux/remoteproc.h
> >>> index da15b77583d3..329c1c071dcf 100644
> >>> --- a/include/linux/remoteproc.h
> >>> +++ b/include/linux/remoteproc.h
> >>> @@ -656,6 +656,7 @@ rproc_of_resm_mem_entry_init(struct device *dev, u32 of_resm_idx, size_t len,
> >>>
> >>> int rproc_boot(struct rproc *rproc);
> >>> void rproc_shutdown(struct rproc *rproc);
> >>> +int rproc_detach(struct rproc *rproc);
> >>> int rproc_set_firmware(struct rproc *rproc, const char *fw_name);
> >>> void rproc_report_crash(struct rproc *rproc, enum rproc_crash_type type);
> >>> int rproc_coredump_add_segment(struct rproc *rproc, dma_addr_t da, size_t size);
> >>>
> >>
> >> From: Arnaud Pouliquen <arnaud.pouliquen@xxxxxxxxxxx>
> >> Date: Tue, 8 Dec 2020 18:54:51 +0100
> >> Subject: [PATCH] remoteproc: core: fix detach for unmapped table_ptr
> >>
> >> If the firmware has been loaded and started by the kernel, the
> >> resource table has probably been mapped by the carveout allocation
> >> (see rproc_elf_find_loaded_rsc_table).
> >> In this case the memory can have been unmapped before the vrings are free.
> >> The result is a crash that occurs in rproc_free_vring while try to use the
> >> unmapped pointer.
> >>
> >> Signed-off-by: Arnaud Pouliquen <arnaud.pouliquen@xxxxxxxxxxx>
> >> ---
> >> drivers/remoteproc/remoteproc_core.c | 17 ++++++++++++++---
> >> 1 file changed, 14 insertions(+), 3 deletions(-)
> >>
> >> diff --git a/drivers/remoteproc/remoteproc_core.c
> >> b/drivers/remoteproc/remoteproc_core.c
> >> index 2b0a52fb3398..3508ffba4a2a 100644
> >> --- a/drivers/remoteproc/remoteproc_core.c
> >> +++ b/drivers/remoteproc/remoteproc_core.c
> >> @@ -1964,6 +1964,13 @@ int rproc_detach(struct rproc *rproc)
> >> goto out;
> >> }
> >>
> >> + /*
> >> + * Prevent case that the installed resource table is no longer
> >> + * accessible (e.g. memory unmapped), use the cache if available
> >> + */
> >> + if (rproc->cached_table)
> >> + rproc->table_ptr = rproc->cached_table;
> >
> > I don't think there is an explicit need to check ->cached_table. If the remote
> > processor has been started by the remoteproc core it is valid anyway. And below
> > kfree() is called invariably.
>
> The condition is needed, the rproc->cached_table is null if the firmware as
> been preloaded and the Linux remote proc just attaches to it.
> The cached is used only when Linux loads the firmware, as the resource table is
> extracted from the elf file to parse resource before the load of the firmware.
I have taken another look at this and you are correct. The if() condition is
needed because ->table_ptr is set only once when the platform driver is
probed. See further down...
>
> >
> > So that problem is fixed. Let me know about your FW image and we'll pick it up
> > from there.
>
> I use the following example available on the stm32mp1 image:
> /usr/local/Cube-M4-examples/STM32MP157C-DK2/Applications/OpenAMP/OpenAMP_TTY_echo_wakeup/lib/firmware/
> This exemple use the RPMsg and also blink a LED when while running.
>
> Don't hesitate if you need me to send it to you by mail.
>
> Thank,
> Arnaud
>
> >
> > Mathieu
> >
> >> +
> >> ret = __rproc_detach(rproc);
> >> if (ret) {
> >> atomic_inc(&rproc->power);
> >> @@ -1975,10 +1982,14 @@ int rproc_detach(struct rproc *rproc)
> >>
> >> rproc_disable_iommu(rproc);
> >>
> >> + /* Free the chached table memory that can has been allocated*/
> >> + kfree(rproc->cached_table);
> >> + rproc->cached_table = NULL;
> >> /*
> >> - * Set the remote processor's table pointer to NULL. Since mapping
> >> - * of the resource table to a virtual address is done in the platform
> >> - * driver, unmapping should also be done there.
> >> + * Set the remote processor's table pointer to NULL. If mapping
> >> + * of the resource table to a virtual address has been done in the
> >> + * platform driver(attachment to an existing firmware),
> >> + * unmapping should also be done there.
> >> */
> >> rproc->table_ptr = NULL;
With the above in mind we can't to that, otherwise trying to re-attach with
rproc_attach() won't work because ->table_ptr will be NULL.
I wasn't able to test that code path because I didn't have the FW that supported
detaching. Now that the feature is maturing it needs to be done.
> >> out:
> >> --
> >> 2.17.1
> >>
> >>
> >>
