There's this piece of code with no comments in arch/i86/mm/malloc.c:
if (currentp->t_begstack > currentp->t_endbrk)
if(len > currentp->t_endseg - 0x1000) {
return -ENOMEM;
}
I don't understand why t_endseg is being reduced by 0x1000 before
checking and can't find any justification for it anywhere else in the
kernel code; fs/exec.c:sys_execve() sets t_endseg and only says "needed
for sys_brk()" which is of no help. Documentation/text/bin_formats.txt
mentions 0x1000 as part of an example while discussing the ELKS
executable file format but it doesn't quite seem to be related. From
what I can gather, ELKS uses 0x1000 as a data segment offset in some
places but I can't see any of those being related to what's going on in
sys_brk()...
One thing is for certain: this check is causing the frustrating problem
with the "login:" prompt that never ends. Removing the offset in the
check fixes the login loop problem (which has plagued ELKS for a long
time) and I can't seem to trigger any bad behavior in any random
userspace programs with the check removed.
The t_endseg check code was originally introduced when ELKS executable
format support was put into CVS in 2003 or so. If anyone knows why the
0x1000 reduction was there, please explain it to me. Otherwise I'm
leaving the change I've already committed in place which fixes this
problem for now so we can worry about other things.
-Jody
--
To unsubscribe from this list: send the line "unsubscribe linux-8086" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
