On Wed, 14 Jun 2023, Nathan Chancellor wrote:
> Hi Hugh,
>
> On Thu, Jun 08, 2023 at 12:17:24PM -0700, Hugh Dickins wrote:
> > Don't make update_mmu_cache() a wrapper around __update_tlb(): call it
> > directly, and use the ptep (or pmdp) provided by the caller, instead of
> > re-calling pte_offset_map() - which would raise a question of whether a
> > pte_unmap() is needed to balance it.
> >
> > Check whether the "ptep" provided by the caller is actually the pmdp,
> > instead of testing pmd_huge(): or test pmd_huge() too and warn if it
> > disagrees? This is "hazardous" territory: needs review and testing.
> >
> > Signed-off-by: Hugh Dickins <hughd@xxxxxxxxxx>
> > ---
> > arch/mips/include/asm/pgtable.h | 15 +++------------
> > arch/mips/mm/tlb-r3k.c | 5 +++--
> > arch/mips/mm/tlb-r4k.c | 9 +++------
> > 3 files changed, 9 insertions(+), 20 deletions(-)
> >
> > diff --git a/arch/mips/include/asm/pgtable.h b/arch/mips/include/asm/pgtable.h
> > index 574fa14ac8b2..9175dfab08d5 100644
> > --- a/arch/mips/include/asm/pgtable.h
> > +++ b/arch/mips/include/asm/pgtable.h
> > @@ -565,15 +565,8 @@ static inline pte_t pte_swp_clear_exclusive(pte_t pte)
> > }
> > #endif
> >
> > -extern void __update_tlb(struct vm_area_struct *vma, unsigned long address,
> > - pte_t pte);
> > -
> > -static inline void update_mmu_cache(struct vm_area_struct *vma,
> > - unsigned long address, pte_t *ptep)
> > -{
> > - pte_t pte = *ptep;
> > - __update_tlb(vma, address, pte);
> > -}
> > +extern void update_mmu_cache(struct vm_area_struct *vma,
> > + unsigned long address, pte_t *ptep);
> >
> > #define __HAVE_ARCH_UPDATE_MMU_TLB
> > #define update_mmu_tlb update_mmu_cache
> > @@ -581,9 +574,7 @@ static inline void update_mmu_cache(struct vm_area_struct *vma,
> > static inline void update_mmu_cache_pmd(struct vm_area_struct *vma,
> > unsigned long address, pmd_t *pmdp)
> > {
> > - pte_t pte = *(pte_t *)pmdp;
> > -
> > - __update_tlb(vma, address, pte);
> > + update_mmu_cache(vma, address, (pte_t *)pmdp);
> > }
> >
> > /*
> > diff --git a/arch/mips/mm/tlb-r3k.c b/arch/mips/mm/tlb-r3k.c
> > index 53dfa2b9316b..e5722cd8dd6d 100644
> > --- a/arch/mips/mm/tlb-r3k.c
> > +++ b/arch/mips/mm/tlb-r3k.c
> > @@ -176,7 +176,8 @@ void local_flush_tlb_page(struct vm_area_struct *vma, unsigned long page)
> > }
> > }
> >
> > -void __update_tlb(struct vm_area_struct *vma, unsigned long address, pte_t pte)
> > +void update_mmu_cache(struct vm_area_struct *vma,
> > + unsigned long address, pte_t *ptep)
> > {
> > unsigned long asid_mask = cpu_asid_mask(¤t_cpu_data);
> > unsigned long flags;
> > @@ -203,7 +204,7 @@ void __update_tlb(struct vm_area_struct *vma, unsigned long address, pte_t pte)
> > BARRIER;
> > tlb_probe();
> > idx = read_c0_index();
> > - write_c0_entrylo0(pte_val(pte));
> > + write_c0_entrylo0(pte_val(*ptep));
> > write_c0_entryhi(address | pid);
> > if (idx < 0) { /* BARRIER */
> > tlb_write_random();
> > diff --git a/arch/mips/mm/tlb-r4k.c b/arch/mips/mm/tlb-r4k.c
> > index 1b939abbe4ca..c96725d17cab 100644
> > --- a/arch/mips/mm/tlb-r4k.c
> > +++ b/arch/mips/mm/tlb-r4k.c
> > @@ -290,14 +290,14 @@ void local_flush_tlb_one(unsigned long page)
> > * updates the TLB with the new pte(s), and another which also checks
> > * for the R4k "end of page" hardware bug and does the needy.
> > */
> > -void __update_tlb(struct vm_area_struct * vma, unsigned long address, pte_t pte)
> > +void update_mmu_cache(struct vm_area_struct *vma,
> > + unsigned long address, pte_t *ptep)
> > {
> > unsigned long flags;
> > pgd_t *pgdp;
> > p4d_t *p4dp;
> > pud_t *pudp;
> > pmd_t *pmdp;
> > - pte_t *ptep;
> > int idx, pid;
> >
> > /*
> > @@ -326,10 +326,9 @@ void __update_tlb(struct vm_area_struct * vma, unsigned long address, pte_t pte)
> > idx = read_c0_index();
> > #ifdef CONFIG_MIPS_HUGE_TLB_SUPPORT
> > /* this could be a huge page */
> > - if (pmd_huge(*pmdp)) {
> > + if (ptep == (pte_t *)pmdp) {
> > unsigned long lo;
> > write_c0_pagemask(PM_HUGE_MASK);
> > - ptep = (pte_t *)pmdp;
> > lo = pte_to_entrylo(pte_val(*ptep));
> > write_c0_entrylo0(lo);
> > write_c0_entrylo1(lo + (HPAGE_SIZE >> 7));
> > @@ -344,8 +343,6 @@ void __update_tlb(struct vm_area_struct * vma, unsigned long address, pte_t pte)
> > } else
> > #endif
> > {
> > - ptep = pte_offset_map(pmdp, address);
> > -
> > #if defined(CONFIG_PHYS_ADDR_T_64BIT) && defined(CONFIG_CPU_MIPS32)
> > #ifdef CONFIG_XPA
> > write_c0_entrylo0(pte_to_entrylo(ptep->pte_high));
> > --
> > 2.35.3
> >
>
> I just bisected a crash while powering down a MIPS machine in QEMU to
> this change as commit 8044511d3893 ("mips: update_mmu_cache() can
> replace __update_tlb()") in linux-next.
Thank you, Nathan, that's very helpful indeed. This patch certainly knew
that it wanted testing, and I'm glad to hear that it is now seeing some.
While powering down? The messages below look like it was just coming up,
but no doubt that's because you were bisecting (or because I'm unfamiliar
with what messages to expect there). It's probably irrelevant information,
but I wonder whether the (V)machine worked well enough for a while before
you first powered down and spotted the problem, or whether it's never got
much further than trying to run init (busybox)? I'm trying to get a feel
for whether the problem occurs under common or uncommon conditions.
> Unfortunately, I can still
> reproduce it with the existing fix you have for this change on the
> mailing list, which is present in next-20230614.
Right, that later fix was only for a build warning, nothing functional
(or at least I hoped that it wasn't making any functional difference).
Thanks a lot for the detailed instructions below: unfortunately, those
would draw me into a realm of testing I've never needed to enter before,
so a lot of time spent on setup and learning. Usually, I just stare at
the source.
What this probably says is that I should revert most my cleanup there,
and keep as close to the existing code as possible. But some change is
needed, and I may need to understand (or have a good guess at) what was
going wrong, to decide what kind of retreat will be successful.
Back to the source for a while: I hope I'll find examples in nearby MIPS
kernel source (and git history), which will hint at the right way forward.
Then send you a patch against next-20230614 to try, when I'm reasonably
confident that it's enough to satisfy my purpose, but likely not to waste
your time.
Thanks, until later,
Hugh
>
> I can reproduce it with the GCC 13.1.0 on kernel.org [1].
>
> $ make -skj"$(nproc)" ARCH=mips CROSS_COMPILE=mips-linux- mrproper malta_defconfig vmlinux
>
> $ qemu-system-mipsel \
> -display none \
> -nodefaults \
> -cpu 24Kf \
> -machine malta \
> -kernel vmlinux \
> -initrd rootfs.cpio \
> -m 512m \
> -serial mon:stdio
> ...
> Linux version 6.4.0-rc6-next-20230614 (nathan@dev-arch.thelio-3990X) (mips-linux-gcc (GCC) 13.1.0, GNU ld (GNU Binutils) 2.40) #1 SMP Wed Jun 14 16:13:02 MST 2023
> ...
> Run /init as init process
> process '/bin/busybox' started with executable stack
> do_page_fault(): sending SIGSEGV to init for invalid read access from 0000003c
> epc = 77b893dc in ld-uClibc-1.0.39.so[77b84000+8000]
> ra = 77b8930c in ld-uClibc-1.0.39.so[77b84000+8000]
> Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
> ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b ]---
>
> The rootfs is available at [2] if it is needed. I am more than happy to
> provide additional information or test patches if necessary.
>
> [1]: https://mirrors.edge.kernel.org/pub/tools/crosstool/
> [2]: https://github.com/ClangBuiltLinux/boot-utils/releases/download/20230609-194440/mipsel-rootfs.cpio.zst
>
> Cheers,
> Nathan
