Recently I noticed a regression when running an old libc5 binary
(amiga-lilo) on m68k. It fails with the error message:
Can't allocate memory
Strace shows that the offending part is:
brk(0x80008e40) = 0x80009000
For older kernels, brk() behaved like:
brk(0x80008e40) = 0x80008e40
brk(0x80009000) = 0x80009000
Also our good old libc5 emergency ramdisk no longer works (init cannot
open /inittab).
At first I suspected CONFIG_COMPAT_BRK, but different values (0, 1, 2) of
/proc/sys/kernel/randomize_va_space don't seem to make any difference.
So I bisected it to:
commit 4cc6028d4040f95cdb590a87db478b42b8be0508
Author: Jiri Kosina <jkosina@xxxxxxx>
Date: Wed Feb 6 22:39:44 2008 +0100
brk: check the lower bound properly
There is a check in sys_brk(), that tries to make sure that we do not
underflow the area that is dedicated to brk heap.
The check is however wrong, as it assumes that brk area starts immediately
after the end of the code (+bss), which is wrong for example in
environments with randomized brk start. The proper way is to check whether
the address is not below the start_brk address.
Signed-off-by: Jiri Kosina <jkosina@xxxxxxx>
Signed-off-by: Ingo Molnar <mingo@xxxxxxx>
diff --git a/mm/mmap.c b/mm/mmap.c
index bb4c963..ad6e4ea 100644
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -245,7 +245,7 @@ asmlinkage unsigned long sys_brk(unsigned long brk)
down_write(&mm->mmap_sem);
- if (brk < mm->end_code)
+ if (brk < mm->start_brk)
goto out;
/*
Reverting this change on current mainline fixes both libc5 amiga-lilo and the
libc5 emergency ramdisk. The value of /proc/sys/kernel/randomize_va_space still
doesn't matter.
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@xxxxxxxxxxxxxx
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
--
To unsubscribe from this list: send the line "unsubscribe linux-m68k" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
