Thu, 16 Nov 2006

Non-recursive Automake.

A lot of people (yeah, you know who you are) bitch about Automake and the associated tools like autoconf and libtool. While I do agree that these tools do have problems and limitations, they are also a better soultion to the problem than any of the alternatives I have looked at.

The thing I really like about automake is that it does automatic dependency checking so that if the file foo.cc includes foo.h which includes bar.h which includes baz.h and baz.h changes, automake knows that foo.c needs to be recompiled. Manually keeping track of dependencies like these is a royal pain in the neck and getting it wrong can lead to really obscure Heisenbugs; for example, two C++ object files disagreeing on the parameter list of a method of a class.

I've had a number of projects that have used automake for years. However, all of these projects used the traditional recursive make scheme where there is a Makefile.am in each directory of the source tree. I continued to do it this way with automake even after reading Peter Miller's excellent paper Recursive Make Considered Harmful, but with hand written Makefiles, I usually took Peter's advice.

My first test of a non-recursive automake solution was for a project I'm doing at work. The project started out with a standard single top level non-recursive Makefile which handled the compiling of about 150 C++ source files which compiled to a couple of static convenience libraries, a main executable and a couple of test programs.

The big problem with the existing standard Makefile was that it didn't properly encode dependancies and hence I often had to do a "make clean" followed by a make to get the thing built correctly. Fixing this issue was the prime motivator for moving to automake.

One slightly unusual aspect of the project was the way project specific internal include files were referenced within the project. As a result of the project having been developed with a single top level Makefile from the beginning, all hash includes within the project are of the form: 


  #include <path/to/header.h>

with "path" being a directory in the same top level directory as the Makefile. What this means is that no source file which includes a header from within the project should need any extra project internal include path other than "-I.". This means that the resulting compile lines produced by automake (and libtool if it is in the picture) are considerably shorter than they would have been otherwise.

So, what does the Makefile.am look like?

Here's an example non-recursive Makefile.am which is basically a stripped down version of the one I'm using on my project but with some extra comments. Anyone who has hacked a Makefile.am before should be able to understand what is going on. 


  # Tell automake to put the object file for apple/apple.c in dir apple/
  AUTOMAKE_OPTIONS := subdir-objects

  # The installable executable.
  bin_PROGRAMS = apple/apple

  # Couple of python scripts used using build.
  EXTRA_DIST = apple/version_create.py apple/tests/test_wrapper.py

  # Convienience libraries required during build.
  noinst_LTLIBRARIES = lib/libcore.la apple/libapple.la

  # All the project related headers required for building.
  noinst_HEADER = $(libcore_includes) $(libapple_includes)

  # Test programs that will not be installed.
  noinst_PROGRAMS = apple/tests/skin_test lib/pip/test/pip_test

  # A couple of autogenerted header files.
  nodist_include_HEADERS = apple/version.h

  DISTCLEANFILES = apple/version.h

  #=========================================================
  # libcore : The core library routines.

  lib_libcore_includes = \
      lib/red.h lib/green.h lib/blue.h

  lib_libcore_la_SOURCES = $(lib_libcore_includes) \
      lib/red.cc lib/green.cc lib/blue.cc

  #=========================================================
  # libpip : All the pips.

  apple_pip_libpip_includes = \
      apple/pip/cat.h apple/pip/dog.h apple/pip/mouse.h

  apple_pip_libpip_la_SOURCES = $(apple_pip_libpip_includes) \
      apple/pip/cat.cc apple/pip/dog.cc apple/pip/mouse.cc

  #=========================================================
  # libapple : Everything in the application except main.cc.

  libapple_includes = \
      apple/granny.h apple/smith.h apple/johnathon.h

  libapple_la_SOURCES = $(libapple_includes) \
      apple/granny.cc apple/smith.cc apple/johnathon.cc

  #=========================================================
  # apple : The application.

  apple_apple_SOURCES = apple/main.cc apple/version.h

  apple_apple_LDADD = apple/libapple.la lib/libcore.la \
      apple/pip/libpip.la $(EXT_A_LIBS) $(EXT_B_LIBS)

  #=========================================================
  # Test programs.

  apple_tests_skin_test_SOURCES = apple/tests/skin_test.cc
  apple_tests_skin_test_LDADD = lib/libcore.la apple/libapple.la

  lib_pip_test_pip_test_SOURCES = lib/pip/test/pip_test.cc
  lib_pip_test_pip_test_LDADD = lib/libcore.la apple/pip/libpip.la \
      $(EXT_A_LIBS) $(EXT_B_LIBS)

  check : $(noinst_PROGRAMS)
      ./apple/tests/skin_test
      ./lib/pip/test/pip_test
      $(top_srcdir)/apple/tests/test_wrapper.py
      @echo
      @echo "All tests passed."
      @echo

  #=========================================================
  # Autogenerated files and their dependancies.

  apple/main.o : apple/version.h
  apple/version.h : this_file_does_not_exist
      $(top_srcdir)/apple/version_create.py $@

  .PHONY : this_file_does_not_exist

The thing that surprised me most about converting this project to automake was how easy it was and how well it worked. I also immediately noticed that the autogenerted non-recursive make seemed to run a lot faster than I was used to with recursive make, but that is one of the benefits mentioned in Peter's paper.

Since this was such a success I'm going to look into applying this to some of my other projects.

Posted at: 21:49 | Category: CodeHacking | Permalink