Tuesday 27 January 2009

[Code] Building old toolchains on a new Linux (disabling Fortify)

We used to build our embedded Linux platform on an aged Debian Linux distribution (a 3-year-old vintage). Reently, I moved the build/release servers and my development workstation to Kubuntu 8.10. Shiny new, and still Linux. What could possibly go wrong?

Well, all the old toolchain buildy stuff stopped working.

The toolchain (which was build with the then-aged crosstool) exploded messily during program linkage, with aborts due to heap corruption or double-freeing. I've been scratching my head over this problem for a while.

The telling symptom in the crash backtraces was the function __fortify_fail just before the abort. The kubuntu glibc has been build with Fortify support (the "best" Fortify homepage I can find is a random geocities one; there must be a better one). Fortify is a memory access checker. Very useful to make sure your code is safe to run.

However, the old toolchain we rely on (specifically, the ar in binutils 2.16.1) seems to be badly behaved. It generates good code, but tramples on its own memory whilst it does it. So Fortify kills the process before an executable sees the light of day.

There are three solutions:
  • Fix the toolchain
  • Port the build system to a newer toolchain
  • Switch off fortify checking
Sadly, I don't have time to dig into debugging an archaic binutils right now; and I'm happy from experience that the code it's generating is good. I've been working on migrating our build to a new set of tools and a newer Linux kernel, but if you've ever played with toolchains and build systems you'll know that it's easier said than done. In fact, it's a nightmare, and I just need to get the existing stuff building again right now.

So option 3, it is then. But without any adequate documentation, it's hard to work out out to disable fortify.

The trick is to define a preprocessor variable _FORTIFY_SOURCE=0. Then all glibc memory checking is disabled and the toolchain can continue to not work as well as it did in the past. A handy trick that's almost obvious when you know where to look for it.

Now, how you get that through crosstool to the binutils build stage is left as an excercise for the reader. Suffice to say that our old crosstool is now incredibly bodged and patched up to just work in 2009.

2 comments:

Anonymous said...

AFAIK the reference for _FORTIFY_SOURCE is this post:
http://gcc.gnu.org/ml/gcc-patches/2004-09/msg02055.html

To build for old targets, I'm currently using these flags:
-fno-stack-protector -D_FORTIFY_SOURCE=0 -Wl,--hash-style=both

Here are some useful links:
- http://ldn.linuxfoundation.org/blog-entry/binary-compatibility-issue-between-linux-distributions-and-versions
- http://avinesh.googlepages.com/binaryincompatibilitybetweenrhel4andrhel
- http://evanjones.ca/portable-linux-binaries.html

Unknown said...

Thanks for the hints Pete!

I added EXTRA_TARGET_CFLAGS="-U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=0" to my .dat file and then CFLAGS="$EXTRA_TARGET_CFLAGS" to the line where binutils is built (make CFLAGS="$EXTRA_TARGET_CFLAGS" $PARALLELMFLAGS all).

My only concern is https://gcc.gnu.org/ml/gcc-help/2011-03/msg00262.html says that FORTIFY_SOURCE should only be used to build GCC and not a library.

Did you make the same change or did you add the CFLAGS to the GCC build steps?