On Thu, 2008-04-17 at 09:40 -0700, Linus Torvalds wrote:
>
> On Thu, 17 Apr 2008, Peter Zijlstra wrote:
> >
> > Here you go ;-)
>
> I think you should _use_ the new functions too ;)
D'0h - clearly not my day today...
Index: linux-2.6/arch/x86/mm/gup.c
===================================================================
--- linux-2.6.orig/arch/x86/mm/gup.c
+++ linux-2.6/arch/x86/mm/gup.c
@@ -9,6 +9,49 @@
#include <linux/vmstat.h>
#include <asm/pgtable.h>
+#ifdef CONFIG_X86_PAE
+
+/*
+ * Companion to native_set_pte_present(); normal access takes the pte_lock
+ * and thus doesn't need it.
+ *
+ * This closes the race:
+ *
+ * CPU#1 CPU#2
+ * ===== =====
+ *
+ * fast_gup:
+ * - read low word
+ *
+ * native_set_pte_present:
+ * - set low word to 0
+ * - set high word to new value
+ *
+ * - read high word
+ *
+ * - set low word to new value
+ *
+ */
+static inline pte_t native_get_pte(pte_t *ptep)
+{
+ pte_t pte;
+
+retry:
+ pte.pte_low = ptep->pte_low;
+ smp_rmb();
+ pte.pte_high = ptep->pte_high;
+ smp_rmb();
+ if (unlikely(pte.pte_low != ptep->pte_low))
+ goto retry;
+ return pte;
+}
+
+#else
+
+#define native_get_pte(ptep) (*(ptep))
+
+#endif
+
/*
* The performance critical leaf functions are made noinline otherwise gcc
* inlines everything into a single function which results in too much
@@ -36,7 +79,7 @@ static noinline int gup_pte_range(pmd_t
* function that will do this properly, so it is broken on
* 32-bit 3-level for the moment.
*/
- pte_t pte = *ptep;
+ pte_t pte = native_get_pte(ptep);
struct page *page;
if ((pte_val(pte) & mask) != result)
--
To unsubscribe from this list: send the line "unsubscribe linux-arch" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
