After several days of hunting, I found a bug in the MIPS highmem code.
A comparison of arch/mips/mm/init.c to arch/i386/mm/init.c supports my
claim.
The PGD entry for the fixed mapping virtual addresses is never
allocated. So what happens is that the fixed mapping pte's get stuffed
into the invalid_pte_table! Then, subsequent accesses that ought to
fault might alias into these PTE's and get satisfied with somebody
else's physical page.
The following patch seems to help a great deal:
Index: arch/mips/mm/init.c
===================================================================
RCS file: /cvs/linux/arch/mips/mm/init.c,v
retrieving revision 1.38.2.4
diff -u -r1.38.2.4 init.c
--- arch/mips/mm/init.c 2002/02/06 18:29:15 1.38.2.4
+++ arch/mips/mm/init.c 2002/03/14 05:25:12
@@ -206,6 +206,12 @@
#ifdef CONFIG_HIGHMEM
/*
+ * Fixed mappings:
+ */
+ vaddr = __fix_to_virt(__end_of_fixed_addresses - 1) & PMD_MASK;
+ fixrange_init(vaddr, 0, pgd_base);
+
+ /*
* Permanent kmaps:
*/
vaddr = PKMAP_BASE;
