Copyright (C) 2003-2005 by David J. Hardy.
All rights reserved.


The MDX Build System
--------------------


Goals:

- separate user projects by directory to avoid development and
  CVS modification conflicts (e.g. mdapi, mdio, force library,
  MG library, PME library, integration library, MTS library,
  MD engine, mdsim front end executable)

- simple project building, with support both for incremental builds
  from within the local project directory and comprehensive builds
  from the outer mdx/ directory

- straightforward to indicate file dependencies and set up local
  makefiles in project directories

- portable among the widest variety of Unix architectures, perhaps
  with eventual Windows support

- support for C and C++ (with possibility of adding Fortran later)

- support for "parallel" builds, meaning use of -j to GNU make and
  also allowing simultaneous compilation towards different target
  architectures from within the same source tree (i.e. nfs mounted
  directory containing source tree from several different machines,
  matching the TCB file system setup)

- easy to configure (at least for a known architecture),
  straightforward to setup for a new architecture

- well-documented so that later bionumerics group members can
  support the build system

- use GNU make extensions so that it is *much* easier to code the
  build system, while being portable to virtually every architecture


Working directory structure:

mdx/ (root of MDX, contains master make files, SRC and DOC lists)

mdx/src/ (contains user projects subdivided by directory, e.g.
    mdapi, mdio, adt, force, integ, mg, pme, mdsim; project directories
    are self-contained with all related .c and .h files in one place)

    An example to demonstrate handling of included header files:
      #include "mdapi/mdtypes.h"
    compile using option -I$(srcdir)

mdx/doc/ (contains documentation for user projects, should use same
    names as corresponding src/ projects)

mdx/build/ (where object files are stored during build, in subdirectories
    according to particular architecture, e.g. BUILD=Linux_generic)

  subdirectories underneath mdx/build/ are

    $(BUILD)/bin/ (binary executables)
    $(BUILD)/lib/ (libraries and plugins)
    $(BUILD)/demo/ (demo executables)
    $(BUILD)/obj_bin/ (.o files for binary executables)
    $(BUILD)/obj_static/ (.o files for static .a libraries)
    $(BUILD)/obj_dynamic/ (.o files for dynamic .so libraries)
    $(BUILD)/obj_plugin/ (.o files for plugins - dlopen()-able modules)
    $(BUILD)/obj_demo/ (.o files for demo executables)

    Note that the obj_ directories also store a .d file for each .o file
    listing the source and header file dependencies for that object.
    These dependency files are generated and maintained automatically.

mdx/arch/ (contains .mk files whose prefix name is same as architecture
    directory, $(BUILD).mk; this file will be included from main makefile)


Configuring:

  make config  # save configuration
  make show    # displays configuration

  (command line options are OS, ARCH, BUILD, BUILD_TYPE, BUILD_DEMO,
  SRC, DOC, INSTALL, INSTALL_DEMO, INSTALL_DIR, INSTALL_BIN, INSTALL_LIB,
  INSTALL_INCLUDE, INSTALL_DOC)

Defaults to BUILD=$(OS)_$(ARCH) if not explicitly set, with defaults
OS=`uname` and ARCH=generic.  (The "generic" option should allow the
code to be built on a typical machine of the specified architecture.)
Attempts to include file mdx/arch/$(BUILD).mk and fails if this file does
not exist.  Stores the configuration options in a hidden file named
mdx/build/.config.`whoami`.`hostname` .  Default values for options:

  OS=`uname`
  ARCH=generic
  BUILD=$(OS)_$(ARCH)
  BUILD_TYPE=dynamic static plugin bin
  BUILD_DEMO=yes
  SRC=`cat src.lst`
  DOC=`cat doc.lst`
  INSTALL=yes
  INSTALL_DEMO=no
  INSTALL_DIR=/usr/local
  INSTALL_BIN=$(INSTALL_DIR)/bin
  INSTALL_LIB=$(INSTALL_DIR)/lib/mdx
  INSTALL_INCLUDE=$(INSTALL_DIR)/include/mdx
  INSTALL_DOC=$(INSTALL_DIR)/doc/mdx

Values for BUILD_TYPE need some explanation.  The library types "dynamic"
and "static" respectively refer to dynamic shared object libraries
(usually .so) and static archive libraries (.a).  The "plugin" type refers
to any modules that are to be dynamically loaded using dlopen() or
equivalent.  Plugins are also installed in the $(INSTALL_LIB) directory.
The "bin" type refers to the principle binary executables.  The BUILD_TYPE
defaults might be overridden by definitions in mdx/arch/$(BUILD).mk in
case certain library types are unsupported on the indicated architecture.
Compilation of demos (i.e. test executables written to demonstrate and/or
stress test isolated MDX modules) is controlled by BUILD_DEMO with their
installation controlled by INSTALL_DEMO.  By default, demos are built but
not installed.

The SRC and DOC options indicate subdirectories in mdx/src/ and mdx/doc/,
respectively, to be built.  The default values are read from files
./src.lst and ./doc.lst.  Success of the source code build is sensitive
to the order of modules presented in ./src.lst indicating library link
dependencies, where the libraries without dependencies should be listed
before the modules that depend upon them.  For instance, if the mdsim
executable depends upon the mdapi library, and if they both depend upon
the adt library, then the listed ordering of these three should be
"adt mdapi mdsim".

It is necessary to know the installation directories before building
because linking the libraries correctly requires knowing their final
location in the file system.  Setting INSTALL=no forces linking to be
done in place (and a later "make install" will be ignored).  INSTALL_DIR
could be set to $(HOME) to install under the user home directory.

Note:  Even though many Unix architectures equate dynamically linked
and shared object libraries (usually .so) with dynamically loadable
modules, not all architectures equate them (e.g. MacOS X).  Also, it
is conceptually better to separate the two, because we want to be
able to build two different versions of, say, an engine, one that is
to be linked against as a library (e.g. libsimen.so) and another that
is dynamically loaded using dlopen() facility (e.g. simen.so).  This
allows use of different predefined constants for conditional compilation
of different sections of code for each.


Building:

  make      # compile and link libraries and executables
  make src  # synonym for "make"
  make doc  # build documentation from latex files (postscript or PDF)
  make all  # same as "make src doc"

It is possible to override the config options.  In particular, it might
be useful to define BUILD_TYPE, BUILD_DEMO, and SRC (for "make" or
"make src") or to define DOC (for "make doc").

Directories in the build subtree are created as needed.


Installing:

  make install

Creates subdirectories and copies files (executables, libraries,
interface/external headers, and documentation) into their respective
directories.  You probably will need root permission to write into
/usr/local/ directory space.

It is possible to override the config options.  In particular, the
definition for SRC determines which interface headers are installed.

For default installation settings, external linking against libraries
requires using the "-L/usr/local/lib/mdx" flag to specify the library
search path, along with the library flags (e.g. -lmdapi).  Including
interface header files requires setting the header file search path
flag "-I/usr/local/include/mdx", and the include statement needs the
module pathname (e.g. #include "mdapi/mdtypes.h").


Cleaning up:

  make clean        # remove binaries, libraries, and object files
  make cleanobj     # remove just the object files

  make cleanconfig  # remove hidden config file

  make cleandoc     # remove PDFs and temp files left from pdflatex
  make cleandoctmp  # remove just the temp files from pdflatex

  make cleanall     # same as "make clean cleandoc cleanconfig"
  make veryclean    # purge ALL files under mdx/build/ plus "make cleandoc"

It is possible to override the config options.  The "cleandoc" and
"cleandoctmp" targets remove pdflatex generated files based on the
definition of DOC.

To free space after build and install use "make cleanobj cleandoctmp".
The "make veryclean" gets rid of ALL generated files.


The "arch" makefiles:

The "arch" directory has makefiles which define architecture dependent
flags needed for compiling MDX on different Unix platforms.  One of
these will be included each time the source code is compiled.  These
files are named $(BUILD).mk, where BUILD is defined by $(OS)_$(ARCH).
The list of flags and their definitions are as follows:

    (these are predefined by GNU make for automatic compilation rules)
  CC  - name of C compiler
  CXX  - name of C++ compiler
  CPPFLAGS  - flags passed to C(C++) preprocessor
    (must at least be set to -I$(srcdir) for correct inclusion of headers)
  CFLAGS  - flags passed to C compiler and linker
  CXXFLAGS  - flags passed to C++ compiler and linker
  TARGET_OS  - architecture dependent flags to C/C++ compilers and linkers
  LDFLAGS  - ?
  LOADLIBES  - ?
  LDLIBS  - ?

    (added for compiling .d dependency files)
  DEPFLAGS  - preprocessor flag to output file dependencies
    (use -M for most compilers, -MM for GNU compiler)

    (for building static libraries)
  slib_name  - static library name, usually: lib$(NAME).a

    (for building dynamic libraries)
  dlib_name  - dynamic library name, usually: lib$(NAME).so
  dlib_flags  - flags appended to CFLAGS and CXXFLAGS just for dynamic libs
  dlib_linkflags  - flags passed to C/C++ linkers

    (for building plugins)
  plug_name  - plugin (dynamically loadable module) name, usually: $(NAME).so
  plug_flags  - flags appended to CFLAGS and CXXFLAGS just for plugins
  plug_linkflags  - flags passed to C/C++ linkers

    (for building binary executables)
  bin_name  - binary executable name, usually: $(NAME)
  bin_linkflags  - flags passed to C/C++ linkers

The predefined variables that might be of use in these definitions:

  NAME  - name of a particular source module, usually the same as the
    the ./src directory name
  srcdir  - absolute path to ./src directory
  libdir  - absolute path to ./build/$(BUILD)/lib directory
    (i.e. where the libs are built before installation)
  INSTALL_LIB  - absolute path to directory where libs will be installed
    (note that when INSTALL != yes, INSTALL_LIB is reset to libdir so
    that library can be linked in place)

The NAME variable is defined in the local makefile of the source module
directory.


The "src" module makefiles:

The ./src directory contains subdirectories, each of which is considered
to be a separate module.  Each module is to build as one or more of a
library, a plugin (dynamically loadable module), or a binary executable.
Additionally, there are two types of libraries supported (depending on
the architecture): "dynamic" (i.e. dynamically linked shared object files,
like the modern ELF library format on modern systems) and "static" (i.e.
the traditional .a archive library format).  Also, there is support to
build "demo" executable(s) from one or more source files, whose purpose
is to demonstrate or test the module.

The design is to allow MDX to be developed in small comprehensible
pieces.  The source file directory layout allows each developer to
work on a few modules separately from the other developers, avoiding
CVS conflicts.  The intention is for each module to have a well-defined
interface so that the modules work together.  It is intended that
executable, plugin, and library modules can depend on other library
modules.

Each ./src directory contains a concise GNUmakefile with definitions
that define the module object dependencies as well as indicating other
library dependencies.  The variables are:

  TOP  - relative path to top level GNUmakefile (should be ../..)
  NAME  - name of this module, usually the same as the directory name
  TYPE  - space separated list of types, any of: lib, plugin, bin
  OBJ  - space separated list of object files that this module depends
    on (no suffixes needed); these will each correspond to a same-named
    C or C++ source file
  LIB  - space separated list of libraries that this module depends on
    (make sure that there are no "lib" prefixes or suffixes; these can
    either be other library modules, e.g. adt, or they can be external
    libraries, e.g. m [the math library] - the -l is appended to each
    entry for inclusion at the end of the command to the linker - note
    also that the listed order is important: since the linker normally
    makes one sequential pass through the libraries, any unresolved
    dependencies must be resolved by the time the last library in the
    list is processed; this means that the math library is generally
    listed *last* since it will never depend on any local libraries;
    also the C standard library should be included automatically by the
    linker at the end of the list since most everything depends on it)
  LINKCPP  - if linking depends on the C++ runtime library, this needs
    to be set to "yes" (without the quotes), otherwise set to "no"
    (even though this will be set automatically if there exists an
    immediately visible C++ source file dependency, it is not easily
    detected if there are dependencies on libraries that themselves
    depend on linking to the C++ runtime library)
  HEADER  - space separated list of header files (no suffixes - assumes
    suffix is always .h) that provide an external interface so need to
    be installed under the INSTALL_INCLUDE directory
  DEMO  - space separated list of demo executables (no suffixes); each
    entry here is to be built into its own executable of the indicated
    name from a same-named C or C++ source file
  DEMOOBJ  - space separated list of demo object files (no suffixes)
    that *all* of the demos listed above depend upon
  DEMOLIB  - space separated list of libraries that the demo(s) depend
    upon (the same notes for LIB apply)
  DEMOLINKCPP  - if linking any of the demo(s) depends on the C++
    runtime library, set to "yes" (without the quotes), otherwise set
    to "no" (the same notes to LINKCPP apply)
  OPTLIB  - space separated list of libraries from LIB that are to be
    considered optional and linked only if they exist; see the section
    "Linking optional libraries" below for more information
  OPTLIBDIR  - space separated list of search directories in which the
    OPTLIB libraries will exist if present; see the section "Linking
    optional libraries" below for more information

If the module GNUmakefile is to allow compiling of the module from
this particular directory, the following four lines must also be
included somewhere in the file (must be before any other rules):

  ifeq (0,$(MAKELEVEL))
  .PHONY : src clean
  src clean : ; $(MAKE) -C $(TOP) SRC="$(NAME)" $@
  endif

This enables a default target (or "make src") that attempts to build
just this module and a "make clean" that removes the object files for
this module alone.

The ./src.mk makefile uses automatic compilation rules of GNU make
to compile source files to object files.  However, since the linking
commands are a bit more involved, ./src.mk defines its own linking
commands (using define .. endef directives).  These commands can be
overridden either locally within the source module GNUmakefile or
globally within the ./arch/$(BUILD).mk file.  The commands are:

  link_static  - create static archive library
  link_dynamic  - link dynamic library
  link_plugin  - link plugin
  link_bin  - link binary executable
  link_demo  - link demo executable

If these commands happen to be defined in both the local and global
included makefiles, since the local makefile is included after the
global makefile, the definition in the local makefile will take
precedence.


Predefined C-preprocessor constants:

The following constants are predefined (appended to CPPFLAGS using
-D the compiler directive) depending on which module TYPE is being
compiled:

  MDX_DYNAMIC  - compiling object files for dynamic libraries
  MDX_STATIC  - compiling object files for static libraries
  MDX_PLUGIN  - compiling object files for plugins
  MDX_BIN  - compiling object files for binary executables

Use of these constants can allow the programmer to introduce changes
in the module depending on, say, whether it is being compiled as a
library or as a plugin.


Linking optional libraries:

The OPTLIB and OPTLIBDIR macros that may be defined in the "src" module
GNUmakefile provide support for linking to optional libraries.  This
permits linking a module to an external library, if present, to take
advantage of its functionality.

The library name needs to already be defined in LIB.  Then having that
name defined in OPTLIB tells the build system that it is an optional
library.  The directories in OPTLIBDIR are searched in the order given
for each OPTLIB library.  If found, then the first match is used when
linking.  Also the conditional macro MDX_OPTLIB_name (where "name" is
replaced by the library name) is expanded to the preprocessor flags
(CPPFLAGS) to indicate to the module source code that the library is
present.  Optional libraries will not be linked against if not found.

For example, suppose that you wish to link your code to the external
DPMTA library, compiled in this case as a sequential FMA solver.
The libraries would, in this case, have full names libspmta.so and
libmpole.so (where spmta depends on mpole).  Suppose that these were
to be expected under your home directory in ~/Linux/lib/ if on Linux
and otherwise not present.  The GNUmakefile might contain the following
lines:

  LIB = spmta mpole m
  OPTLIB = spmta mpole
  OPTLIBDIR = $(HOME)/$(OS)/lib

In this case, $(HOME) expands to your home directory (passed in as an
environmental variable) and $(OS) expands to "Linux" from your
build/.config.* file when compiling on Linux.  If these libraries are
present, then your code will be compiled with -DMDX_OPTLIB_spmta and
-DMDX_OPTLIB_mpole options which expands conditional compilation macros
in your code.  Since these libraries should be used together, you might
want to make sure they are both present in your code, for instance:

  #if defined(MDX_OPTLIB_spmta) && defined(MDX_OPTLIB_mpole)
  #define HAVE_DPMTA
  #else
  #undef HAVE_DPMTA
  #endif

  ...

  #ifdef HAVE_DPMTA
  /* it is safe to call DPMTA routines */
  #else
  /* sorry DPMTA is not available, use an alternate method */
  #endif

This technique also allows optional linking of library modules.  For
example, the mgrid library should not be distributed to the world
before the dissertation is written and papers are submitted, so its
use in other parts of MDX can be made optional.  In this case, the
GNUmakefile might contain the lines:

  LIB = mgrid m
  OPTLIB = mgrid
  OPTLIBDIR =

where if left blank, OPTLIBDIR defaults to $(libdir).

Note that the directories defined by OPTLIBDIR are the only directories
searched for any optional libraries.


The "doc" subdirectory makefiles:

Each subdirectory directly under ./doc contains documentation.
Generally, the documentation is expected to be in LaTeX form, in
which case each directory contributes a single PDF and/or postscript
file.  The variables to be defined in the GNUmakefile are:

  TOP  - relative path to top level GNUmakefile (should be ../..)
  NAME  - name of the document; this is usually chosen to also be
    the name of the subdirectory; there should be a file $(NAME).tex
    that is latex'ed (and/or pdflatex'ed) to generate $(NAME).ps
    (and/or $(NAME).pdf - note that dvips will need to be run to
    convert $(NAME).dvi to $(NAME).ps)
  TYPE  - space separated list of zero or one or both of: ps, pdf.
  DEPS  - space separated list of file dependencies (these *do* need
    suffixes since several different filename extensions might be
    used); the $(NAME).tex file does not need to be listed here

In order to build the documentation while within a particular
subdirectory, the following four lines are needed somewhere in
the GNUmakefile.

  ifeq (0,$(MAKELEVEL))
  .PHONY : doc clean
  doc clean : ; $(MAKE) -C $(TOP) DOC="$(NAME)" $(subst clean,cleandoc,$@)
  endif

This allows "make" (or "make doc"), when issued locally, to rebuild the
documentation and "make clean" to remove the generated files for that
particular document.

Compilation of figures and other related data can be maintained by
augmenting the local GNUmakefile.  Additional files or phony targets
can be added to DEPS, along with further rules to GNUmakefile for
regenerating the figures.  You might want to declare phony targets
using a form like:

  .PHONY : figs

Any phony targets that you use must not already be a target found in
./doc.mk; in particular you cannot use $(NAME) or any of the possible
TYPE values.

The documentation compilation commands can be overridden by the local
GNUmakefile.  The commands and constants defined in ./doc.mk are:

  latex_cmd  - command that calls latex, bibtex, then latex twice more
  pdflatex_cmd  - command that calls pdflatex, bibtex, then pdflatex twice
  dvips_cmd  - command that calls dvips
  dvips_opts  - option to dvips_cmd that sets U.S. letter paper size


Documentation helpful hints:

For some reason, pdflatex defaults to a paper size of 8.3" by 11.7"
(European A4 size?) without any known method to change this to U.S.
letter size of 8.5" by 11".  Including the graphicx package with the
pdftex option:

  \usepackage[pdftex]{graphicx}

changes the output size to U.S. letter.
