On Thu, Jul 13, 2006 at 11:23:22PM -0400, Mike Frysinger wrote:
> we noticed that xterm in Gentoo started to produce broken color display when
> compiled with gcc-4.1 and -Os ... one of our devs ripped apart a small test
> case to illustrate the issue at hand (Debian shows the same issues)
>
> the trouble centers around a matrix declared as static volatile and is never
> initialized ...
Actually, it doesn't matter that it is volatile (it happens to work here
without volatile, but the array in xterm isn't volatile), or that it
isn't explicitly initialised (xterm's array is, and the comment is only
a reminder).
> xterm expects the matrix to be zero set already, but when
> compiled with -Os, this just doesnt work
It doesn't work, but s is initialised correctly. The problem is that the
for loop gets miscompiled. Too many values get printed, and some of them
come from before the start of s.
> example code attached ... to reproduce on an x86 host:
> gcc -Os badcode.c
> or on an amd64 host, just add -m32
> -mike
> #include <stdio.h>
> int main() {
> static volatile char s[256][3]; /* default-initialised to 0 */
> int i;
>
> for(i = 0; i < 256; i++) {
> if(i >= 16 && i < 256)
> printf("%d ", s[i - 16][0]);
> }
> printf("\n");
> }
And the generated assembly code:
.file "test.c"
.section .rodata.str1.1,"aMS",@progbits,1
.LC0:
.string "%d "
.text
.globl main
.type main, @function
main:
pushl %ebx
movl $s.1776-45, %ebx
.L2:
cmpl $s.1776+717, %ebx
ja .L3
movb (%ebx), %al
movsbl %al,%eax
pushl %eax
pushl $.LC0
call printf
popl %edx
popl %ecx
.L3:
addl $3, %ebx
cmpl $s.1776+720, %ebx
jne .L2
pushl $10
call putchar
popl %eax
popl %ebx
ret
.size main, .-main
.local s.1776
.comm s.1776,768,32
.ident "GCC: (GNU) 4.1.1 (Gentoo 4.1.1)"
.section .note.GNU-stack,"",@progbits
The cmpl instruction (at the least) is wrong: without -Os, it uses an unsigned index and starts counting from -15 upwards. -15 is greater (unsigned) than 239, so from -15 ... -1, the printf gets skipped. When this integer comparison is changed to pointer comparison, that logic is no longer true. &s[-15][0] is not likely to compare greater than &s[239][0], and printf is called with garbage data. At least, that's what I think.
