Linux Applications Debugging Techniques/The compiler

When requested to optimize (-O2 and above), the compiler is granted a "total license" to modify it: treats undefined behavior (by the standard) as it cannot happen. Thus, the resulting code behaves differently in release-optimized mode than debug mode. Such differences in behavior cannot be found by static analyzers or code reviews.

Signed Integer Overflow

1. This is C99 undefined behavior and the compiler assumes overflow cannot occur. Thus, any checks for overflow are discarded and the following is an infinite loop in optimized code:

int i, j=0;
for (i = 1; i > 0; i += i) {
    ++j;
}

2. The following:

    (i * 2000) / 1000

is optimized into:

    i * 2

• Use -Wstrict-overflow=N with N>=3 to signal cases where code is optimized away.
• Use -fwrapv

If lucky enough to use gcc 4.9 and later, you can use the ubsan sanitizer:

• compile and link the program with -fsanitize=undefined option
• use other available flags as needed.

Unsigned Wrap Around

The compiler assumes unsigned integers do not wrap. Keep the unsigned variables in the range [INT_MIN/2, INT_MAX/2] to avoid surprises.

"Dead Code" Removed

The memset() call could be removed because the compiler deems buf unused at that point in the code and after:

void do_something(void {
    char buf[123];
    ... use buf...
    /* Clear it. But removed by gcc: */
    memset(buf, 0, sizeof(buf));
}

• Use #pragma optimize() directives to force the code in.
• Use volatile.

volatile Pitfalls

Code can be moved around:

volatile int flag = 0;
char buf[123];
void init_buf() {
    for (size_t i=0; i<123; ++i) {
        buf[i] = 0; //Do something
    }
    flag = 1; // Can move!
}

could be optimized into:

volatile int flag = 0;
char buf[123];
void init_buf() {
    flag = 1; // Moved!
    for (size_t i=0; i<123; ++i) {
        //Do something
    }
}

• Use compiler intrinsic barriers to prevent the flag moving.

Loops could be optimized into one read call only:

void *ptr = address;
while ( *((volatile int*)ptr) & flag ) {}

Pointers

• Use restrict
• Use -fno-delete-null-pointer-checks. Null pointer checks are deleted if placed after the first use of the pointer.

References

• Dangerous Optimizations
• http://blog.regehr.org/archives/918
• LWN about optimizations
• LWN about optimizations (bis)
• cdecl
• Sources of the first C compiler
• gcc internals