On Sat, Apr 10, 2021 at 12:33 AM Michal Suchánek <msuchanek@xxxxxxx> wrote: > > Hello, > > On Fri, Aug 28, 2020 at 04:10:09PM +0800, Pingfan Liu wrote: > > On Thu, Aug 27, 2020 at 3:53 PM Laurent Dufour <ldufour@xxxxxxxxxxxxx> wrote: > > > > > > Le 10/08/2020 à 10:52, Pingfan Liu a écrit : > > > > A bug is observed on pseries by taking the following steps on rhel: > > > > -1. drmgr -c mem -r -q 5 > > > > -2. echo c > /proc/sysrq-trigger > > > > > > > > And then, the failure looks like: > > > > kdump: saving to /sysroot//var/crash/127.0.0.1-2020-01-16-02:06:14/ > > > > kdump: saving vmcore-dmesg.txt > > > > kdump: saving vmcore-dmesg.txt complete > > > > kdump: saving vmcore > > > > Checking for memory holes : [ 0.0 %] / Checking for memory holes : [100.0 %] | Excluding unnecessary pages : [100.0 %] \ Copying data : [ 0.3 %] - eta: 38s[ 44.337636] hash-mmu: mm: Hashing failure ! EA=0x7fffba400000 access=0x8000000000000004 current=makedumpfile > > > > [ 44.337663] hash-mmu: trap=0x300 vsid=0x13a109c ssize=1 base psize=2 psize 2 pte=0xc000000050000504 > > > > [ 44.337677] hash-mmu: mm: Hashing failure ! EA=0x7fffba400000 access=0x8000000000000004 current=makedumpfile > > > > [ 44.337692] hash-mmu: trap=0x300 vsid=0x13a109c ssize=1 base psize=2 psize 2 pte=0xc000000050000504 > > > > [ 44.337708] makedumpfile[469]: unhandled signal 7 at 00007fffba400000 nip 00007fffbbc4d7fc lr 000000011356ca3c code 2 > > > > [ 44.338548] Core dump to |/bin/false pipe failed > > > > /lib/kdump-lib-initramfs.sh: line 98: 469 Bus error $CORE_COLLECTOR /proc/vmcore $_mp/$KDUMP_PATH/$HOST_IP-$DATEDIR/vmcore-incomplete > > > > kdump: saving vmcore failed > > > > > > > > * Root cause * > > > > After analyzing, it turns out that in the current implementation, > > > > when hot-removing lmb, the KOBJ_REMOVE event ejects before the dt updating as > > > > the code __remove_memory() comes before drmem_update_dt(). > > > > So in kdump kernel, when read_from_oldmem() resorts to > > > > pSeries_lpar_hpte_insert() to install hpte, but fails with -2 due to > > > > non-exist pfn. And finally, low_hash_fault() raise SIGBUS to process, as it > > > > can be observed "Bus error" > > > > > > > > From a viewpoint of listener and publisher, the publisher notifies the > > > > listener before data is ready. This introduces a problem where udev > > > > launches kexec-tools (due to KOBJ_REMOVE) and loads a stale dt before > > > > updating. And in capture kernel, makedumpfile will access the memory based > > > > on the stale dt info, and hit a SIGBUS error due to an un-existed lmb. > > > > > > > > * Fix * > > > > This bug is introduced by commit 063b8b1251fd > > > > ("powerpc/pseries/memory-hotplug: Only update DT once per memory DLPAR > > > > request"), which tried to combine all the dt updating into one. > > > > > > > > To fix this issue, meanwhile not to introduce a quadratic runtime > > > > complexity by the model: > > > > dlpar_memory_add_by_count > > > > for_each_drmem_lmb <-- > > > > dlpar_add_lmb > > > > drmem_update_dt(_v1|_v2) > > > > for_each_drmem_lmb <-- > > > > The dt should still be only updated once, and just before the last memory > > > > online/offline event is ejected to user space. Achieve this by tracing the > > > > num of lmb added or removed. > > > > > > > > Signed-off-by: Pingfan Liu <kernelfans@xxxxxxxxx> > > > > Cc: Michael Ellerman <mpe@xxxxxxxxxxxxxx> > > > > Cc: Hari Bathini <hbathini@xxxxxxxxxxxxx> > > > > Cc: Nathan Lynch <nathanl@xxxxxxxxxxxxx> > > > > Cc: Nathan Fontenot <nfont@xxxxxxxxxxxxxxxxxx> > > > > Cc: Laurent Dufour <ldufour@xxxxxxxxxxxxx> > > > > To: linuxppc-dev@xxxxxxxxxxxxxxxx > > > > Cc: kexec@xxxxxxxxxxxxxxxxxxx > > > > --- > > > > v4 -> v5: change dlpar_add_lmb()/dlpar_remove_lmb() prototype to report > > > > whether dt is updated successfully. > > > > Fix a condition boundary check bug > > > > v3 -> v4: resolve a quadratic runtime complexity issue. > > > > This series is applied on next-test branch > > > > arch/powerpc/platforms/pseries/hotplug-memory.c | 102 +++++++++++++++++++----- > > > > 1 file changed, 80 insertions(+), 22 deletions(-) > > > > > > > > diff --git a/arch/powerpc/platforms/pseries/hotplug-memory.c b/arch/powerpc/platforms/pseries/hotplug-memory.c > > > > index 46cbcd1..1567d9f 100644 > > > > --- a/arch/powerpc/platforms/pseries/hotplug-memory.c > > > > +++ b/arch/powerpc/platforms/pseries/hotplug-memory.c > > > > @@ -350,13 +350,22 @@ static bool lmb_is_removable(struct drmem_lmb *lmb) > > > > return true; > > > > } > > > > > > > > -static int dlpar_add_lmb(struct drmem_lmb *); > > > > +enum dt_update_status { > > > > + DT_NOUPDATE, > > > > + DT_TOUPDATE, > > > > + DT_UPDATED, > > > > +}; > > > > + > > > > +/* "*dt_update" returns DT_UPDATED if updated */ > > > > +static int dlpar_add_lmb(struct drmem_lmb *lmb, > > > > + enum dt_update_status *dt_update); > > > > > > > > -static int dlpar_remove_lmb(struct drmem_lmb *lmb) > > > > +static int dlpar_remove_lmb(struct drmem_lmb *lmb, > > > > + enum dt_update_status *dt_update) > > > > { > > > > unsigned long block_sz; > > > > phys_addr_t base_addr; > > > > - int rc, nid; > > > > + int rc, ret, nid; > > > > > > > > if (!lmb_is_removable(lmb)) > > > > return -EINVAL; > > > > @@ -372,6 +381,13 @@ static int dlpar_remove_lmb(struct drmem_lmb *lmb) > > > > invalidate_lmb_associativity_index(lmb); > > > > lmb_clear_nid(lmb); > > > > lmb->flags &= ~DRCONF_MEM_ASSIGNED; > > > > + if (*dt_update) { > > Original, I plan to use it to exclude the case of DT_NOUPDATE, which is value 0. > > And I think it looks better by using if (*dt_update == DT_TOUPDATE) > > > > > > That test is wrong, you should do: > > > if (*dt_update && *dt_update == DT_TOUPDATE) { > > I think you mean if (dt_update && *dt_update == DT_TOUPDATE) { > > > > > > With the current code, the device tree is updated all the time. > > > > > > Another option would be to pass a valid pointer (!= NULL) only when DT update is > > > required, this way you don't need the DT_TOUPDATE value. The caller would have > > > to set the pointer accordingly. The advantage with this option is the caller is > > > guaranteed that its variable is not touched by the callee when no device tree is > > > requested. A simple boolean pointer would be enough without the need to this enum. > > It is expected that dlpar_remove_lmb/dlpar_add_lmb can report whether > > they successfully update dt or not. So the caller can handle the > > different cases. > > Is there any plan to refresh this patch to apply to master? > Spreading the failure and rollback logic around the code looks urgly. Even myself dislike it. But I have not found a better way out. > I am using an older revision of this patch so I am not in the position > to repost an updated version. > > I lack some otimization in my patch so I probably have the quadratic > coplexity of the add mentioned above. > Do you have any good ideas? Thanks, Pingfan _______________________________________________ kexec mailing list kexec@xxxxxxxxxxxxxxxxxxx http://lists.infradead.org/mailman/listinfo/kexec