Thank you!
On Tue, Feb 18, 2025 at 3:51 PM gaoxu <gaoxu2@xxxxxxxxx> wrote:
>
> >
> > On Sat, Feb 15, 2025 at 10:05 PM gaoxu <gaoxu2@xxxxxxxxx> wrote:
> > >
> > > Add a NULL check on the return value of swp_swap_info in
> > > __swap_duplicate to prevent crashes caused by NULL pointer dereference.
> > >
> > > The reason why swp_swap_info() returns NULL is unclear; it may be due
> > > to CPU cache issues or DDR bit flips. The probability of this issue is
> > > very small, and the stack info we encountered is as follows：
> > > Unable to handle kernel NULL pointer dereference at virtual address
> > > 0000000000000058
> > > [RB/E]rb_sreason_str_set: sreason_str set null_pointer Mem abort info:
> > > ESR = 0x0000000096000005
> > > EC = 0x25: DABT (current EL), IL = 32 bits
> > > SET = 0, FnV = 0
> > > EA = 0, S1PTW = 0
> > > FSC = 0x05: level 1 translation fault Data abort info:
> > > ISV = 0, ISS = 0x00000005, ISS2 = 0x00000000
> > > CM = 0, WnR = 0, TnD = 0, TagAccess = 0
> > > GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0 user pgtable: 4k pages,
> > > 39-bit VAs, pgdp=00000008a80e5000 [0000000000000058]
> > > pgd=0000000000000000, p4d=0000000000000000,
> > > pud=0000000000000000
> > > Internal error: Oops: 0000000096000005 [#1] PREEMPT SMP Skip md ftrace
> > > buffer dump for: 0x1609e0 ...
> > > pc : swap_duplicate+0x44/0x164
> > > lr : copy_page_range+0x508/0x1e78
> > > sp : ffffffc0f2a699e0
> > > x29: ffffffc0f2a699e0 x28: ffffff8a5b28d388 x27: ffffff8b06603388
> > > x26: ffffffdf7291fe70 x25: 0000000000000006 x24: 0000000000100073
> > > x23: 00000000002d2d2f x22: 0000000000000008 x21: 0000000000000000
> > > x20: 00000000002d2d2f x19: 18000000002d2d2f x18: ffffffdf726faec0
> > > x17: 0000000000000000 x16: 0010000000000001 x15: 0040000000000001
> > > x14: 0400000000000001 x13: ff7ffffffffffb7f x12: ffeffffffffffbff
> > > x11: ffffff8a5c7e1898 x10: 0000000000000018 x9 : 0000000000000006
> > > x8 : 1800000000000000 x7 : 0000000000000000 x6 : ffffff8057c01f10
> > > x5 : 000000000000a318 x4 : 0000000000000000 x3 : 0000000000000000
> > > x2 : 0000006daf200000 x1 : 0000000000000001 x0 : 18000000002d2d2f Call
> > > trace:
> > > swap_duplicate+0x44/0x164
> > > copy_page_range+0x508/0x1e78
> >
> > This is really strange since we already have a swap entry check before calling
> > swap_duplicate().
> >
> > copy_nonpresent_pte(struct mm_struct *dst_mm, struct mm_struct *src_mm,
> > pte_t *dst_pte, pte_t *src_pte, struct vm_area_struct
> > *dst_vma,
> > struct vm_area_struct *src_vma, unsigned long addr, int
> > *rss) {
> > unsigned long vm_flags = dst_vma->vm_flags;
> > pte_t orig_pte = ptep_get(src_pte);
> > pte_t pte = orig_pte;
> > struct folio *folio;
> > struct page *page;
> > swp_entry_t entry = pte_to_swp_entry(orig_pte);
> >
> > if (likely(!non_swap_entry(entry))) {
> > if (swap_duplicate(entry) < 0)
> > return -EIO;
> > ...
> > }
> >
> > likely the swap_type is larger than MAX_SWAPFILES so we get a NULL?
> >
> > static struct swap_info_struct *swap_type_to_swap_info(int type) {
> > if (type >= MAX_SWAPFILES)
> > return NULL;
> >
> > return READ_ONCE(swap_info[type]); /* rcu_dereference() */ }
> >
> > But non_swap_entry() guarantees that swp_type is smaller than
> > MAX_SWAPFILES.
> >
> > static inline int non_swap_entry(swp_entry_t entry) {
> > return swp_type(entry) >= MAX_SWAPFILES; }
> >
> > So another possibility is that we have an overflow of swap_info[] where type is <
> > MAX_SWAPFILES but is not a valid existing swapfile?
> In the log of this issue, there is a printed entry: get_swap_device:
> Bad swap file entry 18000000002d2d2f.
> It can be calculated that swp_type(18000000002d2d2f) = 6.
> In the Android 15-linux6.6:
> system: MAX_SWAPFILES = 28, nr_swapfiles = 1.
> Since swp_type(18000000002d2d2f)=6 is less than MAX_SWAPFILES but greater
> than nr_swapfiles, the value of this entry is abnormal.
>
> static unsigned int nr_swapfiles;
> static struct swap_info_struct *swap_info[MAX_SWAPFILES];
> swap_info is a static array, with its values initialized to 0.
> The size of the array is MAX_SWAPFILES, and the size of valid values in the array is
> nr_swapfiles. Therefore, when we validate the validity of swp_type(entry),
> we should compare it with nr_swapfiles, not MAX_SWAPFILES.
> The code for validating swp_type may need to be modified as follows:
That might be true, but on a normal system, we only need to distinguish
between a swap entry and a migrate entry. Therefore, comparing with
MAX_SWAPFILES is sufficient.
> static inline int non_swap_entry(swp_entry_t entry)
> {
> - return swp_type(entry) >= MAX_SWAPFILES;
> + return swp_type(entry) >= nr_swapfiles;
> }
>
> static struct swap_info_struct *swap_type_to_swap_info(int type)
> {
> - if (type >= MAX_SWAPFILES)
> + if (type >= nr_swapfiles)
> return NULL;
>
> return READ_ONCE(swap_info[type]); /* rcu_dereference() */
> }
> >
> > I don't see how the current patch contributes to debugging or fixing anything
> > related to this dumped stack. Can we dump swp_type() as well?
> >
> > > copy_process+0x1278/0x21cc
> > > kernel_clone+0x90/0x438
> > > __arm64_sys_clone+0x5c/0x8c
> > > invoke_syscall+0x58/0x110
> > > do_el0_svc+0x8c/0xe0
> > > el0_svc+0x38/0x9c
> > > el0t_64_sync_handler+0x44/0xec
> > > el0t_64_sync+0x1a8/0x1ac
> > > Code: 9139c35a 71006f3f 54000568 f8797b55 (f9402ea8) ---[ end trace
> > > 0000000000000000 ]--- Kernel panic - not syncing: Oops: Fatal
> > > exception
> > > SMP: stopping secondary CPUs
> > >
> > > The patch seems to only provide a workaround, but there are no more
> > > effective software solutions to handle the bit flips problem. This
> > > path will change the issue from a system crash to a process exception,
> > > thereby reducing the impact on the entire machine.
> > >
> > > Signed-off-by: gao xu <gaoxu2@xxxxxxxxx>
> > > ---
> > > v1 -> v2:
> > > - Add WARN_ON_ONCE.
> > > - update the commit info.
> > > v2 -> v3: Delete the review tags (This is my issue, and I apologize).
> > > ---
> > >
> > > mm/swapfile.c | 2 ++
> > > 1 file changed, 2 insertions(+)
> > >
> > > diff --git a/mm/swapfile.c b/mm/swapfile.c index 7448a3876..a0bfdba94
> > > 100644
> > > --- a/mm/swapfile.c
> > > +++ b/mm/swapfile.c
> > > @@ -3521,6 +3521,8 @@ static int __swap_duplicate(swp_entry_t entry,
> > unsigned char usage, int nr)
> > > int err, i;
> > >
> > > si = swp_swap_info(entry);
> > > + if (WARN_ON_ONCE(!si))
> >
> > I mean, printk something related to swp_type(). This is really strange, but the
> > current stack won't help with debugging.
> The log can find info related to "get_swap_device: Bad swap file entry xxx"
> when an entry encounters an exception.
> Add a print info log like the following:
> pr_err("%s%08d\n", Bad swap type, swp_type(entry));
This is really strange. It would be better to have the entire PTE value
dumped so we can determine if a bit-flip occurred on critical bits like
PTE_PRESENT.
In that case, a present PTE could be misinterpreted as a swap entry.
On arm64,
/*
* Encode and decode a swap entry:
* bits 0-1: present (must be zero)
* bits 2: remember PG_anon_exclusive
* bits 3-7: swap type
* bits 8-57: swap offset
* bit 58: PTE_PROT_NONE (must be zero)
*/
#define __SWP_TYPE_SHIFT 3
#define __SWP_TYPE_BITS 5
#define __SWP_OFFSET_BITS 50
#define __SWP_TYPE_MASK ((1 << __SWP_TYPE_BITS) - 1)
#define __SWP_OFFSET_SHIFT (__SWP_TYPE_BITS + __SWP_TYPE_SHIFT)
#define __SWP_OFFSET_MASK ((1UL << __SWP_OFFSET_BITS) - 1)
_swp_type is bits3-7.
For a present pte, bits 3-7 are:
AP[7-6], NS[5], AttributeIndex[4-2].
> >
> > > + return -EINVAL;
> > >
> > > offset = swp_offset(entry);
> > > VM_WARN_ON(nr > SWAPFILE_CLUSTER - offset %
> > SWAPFILE_CLUSTER);
> > > --
> > > 2.17.1
Thanks
Barry
