Building Gentoo Linux with GCC 4.7 and LTO

GCC 4.7 was recently added to portage. It's still hard masked and not keyworded, but works fairly well. As with almost every new major release of GCC, a few packages fail to build, but the community is already working on it; most reported packages have been fixed already. I fixed the ones I encountered myself (kdocker and zsnes) and submitted patches to Gentoo's Bugzilla. Still, if you're using a package that hasn't been fixed yet and no workaround has been posted, you might want to hold off from switching your system to GCC 4.7 as its main compiler. (Just installing 4.7 will have no impact on your system, since the current active version will remain the default.)

There's one feature in this release that stands out: the much improved link time optimization (LTO). What this does is hold off most of the optimization work until the final link step so that the optimizer can work across the whole program rather than on individual object files. LTO is activated using the -flto and -fno-use-linker-plugin options.

As you can guess, this isn't your usual Gentoo ricer option (see -funroll-loops for that). It's a useful feature that helps the optimizer produce much better results. Most commercial compilers had this for a long time but GCC was lagging behind. Well, for now it's a ricer option, since it's unsupported by the Gentoo devs, but I fully expect it to become a supported option at some point as GCC will keep improving it. Building your whole system with LTO was still a hopeless endeavor with GCC 4.6. But in 4.7 it actually improved to the point where you can emerge -e world with it. And I did. And it works, even.

My Gentoo box is a KDE desktop system, with 1043 packages installed. Out of those 1043, only 33 could not be built with LTO. Quite a step up from GCC 4.6! Last time I experimented with it on 4.6, I gave up after about the 100th package failing to build. Worse, some packages would build but not run. So LTO in GCC 4.7 seems to really have improved a lot. There was only one package that would build but not run correctly (dev-libs/glib).

How to disable LTO

So here's what you do if you want to try this yourself. The most important thing to set up is a way to disable LTO for packages that don't work well with it. Fortunately, package.env makes this easy. First, create the file:

/etc/portage/env/no-lto.conf

With the following contents:

CFLAGS="${CFLAGS} -fno-lto -fno-use-linker-plugin"
CXXFLAGS="${CXXFLAGS} -fno-lto -fno-use-linker-plugin"
LDFLAGS="${LDFLAGS} -fno-lto -fno-use-linker-plugin"

You can now use that file for every package that breaks with LTO. You do that by listing the appropriate package atoms, followed by no-lto.conf in /etc/portage/package.env. Here's how that looks on my system:

sys-apps/sysvinit no-lto.conf
dev-lang/perl no-lto.conf
sys-libs/gpm no-lto.conf
dev-libs/elfutils no-lto.conf
>=dev-lang/python-3 no-lto.conf
sys-fs/e2fsprogs no-lto.conf
sys-apps/hdparm no-lto.conf
sys-apps/pciutils no-lto.conf
media-sound/wavpack no-lto.conf
media-libs/libpostproc no-lto.conf
dev-vcs/cvs no-lto.conf
x11-libs/qt-script no-lto.conf
media-libs/alsa-lib no-lto.conf
dev-util/dialog no-lto.conf
sys-apps/hwinfo no-lto.conf
dev-util/valgrind no-lto.conf
app-cdr/cdrtools no-lto.conf
dev-libs/boost no-lto.conf
media-video/libav no-lto.conf
app-text/aspell no-lto.conf
net-misc/nx no-lto.conf
app-text/dvisvgm no-lto.conf
x11-libs/wxGTK no-lto.conf
media-video/mplayer2 no-lto.conf
x11-libs/qt-webkit no-lto.conf
x11-libs/qt-declarative no-lto.conf
x11-base/xorg-server no-lto.conf
dev-tex/luatex no-lto.conf
app-misc/strigi no-lto.conf
kde-base/kdelibs no-lto.conf
kde-base/okular no-lto.conf
net-im/amsn no-lto.conf
sys-devel/llvm no-lto.conf
sys-devel/clang no-lto.conf
app-office/lyx no-lto.conf
dev-libs/glib no-lto.conf
sys-auth/polkit no-lto.conf
net-analyzer/nmap no-lto.conf

Now all of the above packages will be built without LTO. Go ahead and copy the above into your own package.env.

Update: This list is somewhat old now; I suspect many of the above packages have already been fixed or current GCC versions are able to build them without errors.

How to enable LTO

To actually enable LTO, you need to change your make.conf and add -flto -fuse-linker-plugin to your CFLAGS/CXXFLAGS and LDFLAGS. Yes, it's also needed in LDFLAGS. Do not omit it!

In order to speed up LTO-enabled builds, you can pass the amount of concurrent jobs to be performed by the linker to the -flto option. So on a quad core CPU, you'd use -flto=4 (with a BFS kernel) or -flto=5 (with a vanilla kernel.) In general, use the same value from your MAKEOPTS variable.

Note that with LTO, you need to include your optimization flags in LDFLAGS as well. You can use the same optimizations you have in your CFLAGS. Though you can use different ones if you really want to.

Keep in mind that you'll need at least sys-devel/binutils-2.21 for LTO to work correctly. If (for some weird reason) you're still on 2.20, it will not work, since GNU ld versions prior to 2.21 do not support linker plugins (-fuse-linker-plugin).

For reference, here are the relevant entries from my own make.conf:

CFLAGS="-pipe -mtune=native -march=native -O2 -flto=4 -fuse-linker-plugin -fomit-frame-pointer -floop-interchange -floop-strip-mine -floop-block"
CXXFLAGS="${CFLAGS}"
LDFLAGS="-Wl,--as-needed -Wl,-O1 -Wl,--hash-style=gnu -Wl,--sort-common ${CFLAGS}"

You're now ready to emerge -e @system followed by emerge -e @world. Of course you need to make GCC 4.7 the default compiler first. You do that by using the gcc-config tool.

Dealing with breakage

Since you most probably will have a different set of packages installed on your system compared to me, you might encounter build failures. Those can either be caused by LTO or by GCC 4.7 in general. So if a package fails to emerge, add it to package.env with no-lto.conf and emerge --resume. If that fails again for the same package, then it's probably a GCC 4.7 problem. In that case, you can skip building that particular package with emerge --resume --skipfirst. Don't worry, the package will continue to work even with the rest of the system being built with GCC 4.7. GCC 4.7 is compatible with 4.6 and binaries (including libraries) can be built with one and work OK with the other.

It would be nice if you filed a bug for packages that don't build with 4.7 and make that bug block 390247. Note: only file bugs for packages that fail with 4.7 without LTO. If a package only fails when LTO is enabled, don't file a bug; the Gentoo devs are not interested about LTO-related bugs at the moment.

Final thoughts

If you want to ask whether the system runs any faster now, I'm not going to answer that. I didn't run any benchmarks, and statements like "this and that program feel faster now" are subjective to begin with and very prone to placebo. Keep in mind that I didn't rebuild the whole system in order to get a faster system. The system was already fast enough. I mainly wanted to test whether LTO is ready for prime time. And it's very close to being ready. The next GCC version will surely improve compatibility even further. One day, package.env might be empty, even :-)