Third-Party Libraries

Jump to a library :

  • Boost
  • Eigen
  • hdf5
  • Silo
  • muParser
  • MPICH
  • Hypre
  • PETSc
  • SAMRAI
  • libMesh
  • IBAMR

    • These instructions describe building and installing IBAMR and its required third-party libraries on a macOS system. The instructions below create both a debug build, where the libraries are compiled with additional checking and can be used with a debugger, and an optimized build, which is generally several times faster than the debug build because most error checking is turned off and debugging information is not available.

    All libraries specified below are compiled with the make program. We recommend running make in parallel whenever possible: make is smart enough to compile multiple files at once. One can specify the number of jobs, or concurrent copies of make to run, with the -j flag: i.e., running make -j4 will compile four files at once. We recommend using the same number of make jobs as physical processors on a machine. The sample instructions here use four jobs.

    The package manager HomeBrew is used here to install several packages IBAMR depends on. After installing HomeBrew, install necessary packages:

    $ brew install autoconf automake boost clang-format gawk gcc hdf5 libtool mpich

    At the present time IBAMR is not compatible with the version of hypre installed through homebrew, so hypre (and PETSc) must be installed manually. You may also install these optional packages:

    $ brew install ccache cmake doxygen gnu-units

    Note: You do not need to build your own versions of Boost, HDF5, or MPICH. All of these packages are now installed, by default, in /usr/local.

    It will be useful to install the wget command through HomeBrew:

    $ brew install wget

    Picking good optimization flags

    IBAMR’s performance in release mode is very sensitive to the choice of optimization flags. For example: switching from -O3 to -march=native -O3 roughly doubled the performance of some IBFE-based codes. This is because a large fraction of the time spent in IBAMR is spent doing structured grid calculations that can benefit heavily from applying processor-specific optimizations.

    If you want good performance then we recommend adding -march=native to the flags chosen below for optimization purposes (i.e., any place where we write -O3, we recommend you use -O3 -march=native). This is not listed as the default compiler flag because it generates processor-specific instructions, which may not be valid if you compile on one machine and execute on another (e.g., if you compile on a cluster’s login node and then run on a partition that uses a different set of hardware). For the most part this is not relevant on a mac (since these are not usually networked) unless you are doing something unusual.

    Most of the time spent by IBAMR applications is spent inside IBAMR functions, so most of the benefit of processor-specific compilation flags is achieved by compiling IBAMR itself (i.e., it is not necessary to recompile dependencies with -march=native, but it can only help).

    For more information see The GCC page on compiler flags. The flag -march=native is supported on all major C++ compilers (GCC, Clang, and Intel).

    Setup

    cd $HOME
mkdir sfw
cd sfw

    Boost

    Currently, IBAMR requires Boost version greater than or equal to 1.57.

    This is available in /usr/local through HomeBrew.

    Eigen

    Currently, IBAMR requires Eigen version greater than or equal to 3.2.5.

    NOTE: Eigen 3.2.5 is bundled with IBAMR and is built automatically by IBAMR configure. It is also possible to use an external installation of Eigen with IBAMR. If you will be using libMesh, you must use an external Eigen.

    HDF5

    any version of HDF5 >= 1.8.7 is likely to work with IBAMR.

    This is available in /usr/local through HomeBrew.

    Silo

    Silo is an optional dependency of IBAMR used to save IB data for visualization. If used, IBAMR requires using the most recent version of Silo (4.11).

    cd $HOME/sfw/
tar xvfz path/to/silo-4.11-bsd.tar.gz
cd silo-4.11-bsd
./configure \
  CC=gcc \
  CXX=g++ \
  FC=gfortran \
  F77=gfortran \
  --enable-optimization \
  --enable-shared=yes \
  --enable-browser=no \
  --with-hdf5=no \
  --with-sz=no \
  --prefix=$HOME/sfw/linux/silo/4.11 
make -j4
make -j4 install

    muParser

    NOTE: A patched version of muParser 2.2.3 is bundled with IBAMR and installed automatically by IBAMR configure. It is also possible to use an external installation of muParser with IBAMR.

    MPI

    IBAMR requires that MPI be available but does not require the use of a specific MPI library. We recommend using a system-provided MPI installation if available. If you will be using your system MPI, wherever you see the path to the MPICH compilers specified, replace it with the path to your system’s MPI compilers. If you do not have a system MPI installation available, we recommend using MPICH. MPICH is available through HomeBrew in /usr/local.

    Hypre

    IBAMR requires Hypre verion 2.10b or later. We recommend you let PETSc install hypre.

    PETSc

    IBAMR requires the PETSc (portable toolkit for scientific computing) library. IBAMR is compatible with PETSc versions 3.7 and newer. If you use a new version of PETSc then you will also need to use a new version of libMesh. These instructions are for PETSc version 3.17.5.

    These configure options direct PETSc to download and install hypre. First, unpack PETSc:

    cd $HOME/sfw
mkdir petsc
cd petsc
tar xvfz petsc-3.17.5.tar.gz
mv petsc-3.17.5 3.17.5
cd 3.17.5

    Next, configure and compile a debugging build of PETSc:

    (Note that if you are using your system MPI, you must modify the following to reflect the correct path to your MPI compilers)

    cd $HOME/sfw/petsc/3.17.5
export PETSC_DIR=$PWD
export PETSC_ARCH=mac-debug
./configure \
    --CC=/usr/local/bin/mpicc \
    --CXX=/usr/local/bin/mpicxx \
    --FC=/usr/local/bin/mpif90 \
    --PETSC_ARCH=$PETSC_ARCH \
    --with-debugging=1 \
    --download-hypre=1 \
    --with-x=0
make -j4
make -j4 test

    Finally, configure and compile an optimized build of PETSc:

    (Note that if you are using your system MPI, you must modify the following to reflect the correct path to your MPI compilers)

    export PETSC_DIR=$PWD
export PETSC_ARCH=mac-opt
./configure \
 --CC=/usr/local/bin/mpicc \
 --CXX=/usr/local/bin/mpicxx \
 --FC=/usr/local/bin/mpif90 \
 --PETSC_ARCH=$PETSC_ARCH \
 --COPTFLAGS="-O3" \
 --CXXOPTFLAGS="-O3" \
 --FOPTFLAGS="-O3" \
 --with-debugging=0 \
 --download-hypre=1 \
 --with-x=0
make -j4
make -j4 test

    NOTE: You may wish to add the lines:

    export PETSC_DIR=$HOME/sfw/petsc/3.17.5
export PETSC_ARCH=mac-opt

    to your ~/.bashrc file.

    SAMRAI

    IBAMR requires SAMRAI version 2.4.4 and does not work with newer (or older) versions of SAMRAI.

    First, download SAMRAI (shown here via wget) and unpack it:

    cd $HOME/sfw
mkdir samrai
cd samrai
mkdir 2.4.4
cd 2.4.4
wget https://github.com/LLNL/SAMRAI/archive/refs/tags/2.4.4.tar.gz
tar xf 2.4.4.tar.gz

    Next, patch SAMRAI using the patch file samrai-2.4.4-patch-ibamr-0.13.patch

    cd SAMRAI-2.4.4
wget https://github.com/IBAMR/IBAMR/releases/download/v0.13.0/samrai-2.4.4-patch-ibamr-0.13.0.patch
patch -p1 < samrai-2.4.4-patch-ibamr-0.13.0.patch

    Next, configure and compile a debugging build of SAMRAI. It may be necessary, if you are not using your system installation of MPI, to modify the paths to the MPI compiler wrappers:

    cd $HOME/sfw/samrai/2.4.4
mkdir objs-debug
cd objs-debug

../SAMRAI-2.4.4/configure \
  CFLAGS="-fPIC" \
  CXXFLAGS="-fPIC" \
  FFLAGS="-fPIC" \
  --prefix=$HOME/sfw/samrai/2.4.4/mac-g++-debug \
  --with-CC=/usr/local/bin/mpicc \
  --with-CXX=/usr/local/bin/mpicxx \
  --with-F77=mpifort \
  --with-hdf5=/usr/local \
  --without-petsc \
  --without-hypre \
  --with-silo=$HOME/sfw/silo/4.11 \
  --without-blaslapack \
  --without-cubes \
  --without-eleven \
  --without-kinsol \
  --without-sundials \
  --without-x \
  --with-doxygen \
  --with-dot \
  --enable-debug \
  --disable-opt \
  --enable-implicit-template-instantiation \
  --disable-deprecated
make -j4
make -j4 install

    Finally, configure and compile an optimized build of SAMRAI:

    cd $HOME/sfw/samrai/2.4.4
mkdir objs-opt
cd objs-opt

../SAMRAI-2.4.4/configure \
  CFLAGS="-fPIC -O3" \
  CXXFLAGS="-fPIC -O3" \
  FFLAGS="-fPIC -O3" \
  --prefix=$HOME/sfw/samrai/2.4.4/mac-g++-opt \
  --with-CC=/usr/local/bin/mpicc \
  --with-CXX=/usr/local/bin/mpicxx \
  --with-F77=mpifort \
  --with-hdf5=/usr/local \
  --without-petsc \
  --without-hypre \
  --with-silo=$HOME/sfw/silo/4.11 \
  --without-blaslapack \
  --without-cubes \
  --without-eleven \
  --without-kinsol \
  --without-sundials \
  --without-x \
  --with-doxygen \
  --with-dot \
  --disable-debug \
  --enable-opt \
  --enable-implicit-template-instantiation \
  --disable-deprecated
  make -j4
  make -j4 install

    Note that, in both cases, we need to add the -fPIC flag so that SAMRAI is compatible with shared object libraries. Strictly speaking, you don’t need this flag if you are using the autotools-based build system for IBAMR, but you do if you use CMake.

    libMesh

    libMesh is a finite element library required by the IBFE subsystem. It is not a required dependency of IBAMR but is strongly recommended.

    IBAMR is compatible with libMesh 1.2.0 or newer. The instructions below use the latest version of libMesh (1.6.2). If you use libMesh with boost, then you MUST use an external build of boost for both IBAMR and libMesh since the subset of boost included with libMesh does not contain everything that IBAMR needs. We recommend compiling libMesh without boost to avoid this issue.

    We recommend compiling libMesh with ExodusII. ExodusII support is enabled by the --enable-exodus flag, which is included below. Many IBAMR examples use libMesh’s ExodusII writer to output data for visualization. The examples can be run without ExodusII but will need to be modified to generate output in another format.

    NOTE: libMesh assumes that PETSc was installed in place, i.e., without calling make install at the end. The instructions on this page assume that PETSc is installed in place. However, if you choose instead to run make install, then one should provide the full path to the PETSc installation as PETSC_DIR: e.g., in this case, one would provide PETSC_DIR=$HOME/sfw/petsc/3.17.5/mac-debug as an argument.

    Furthermore, to work around a bug in libMesh, we have to pass the path to PETSc in CFLAGS so that a configuration check passes.

    cd $HOME/sfw/
mkdir libmesh
cd libmesh
mkdir 1.6.2
cd 1.6.2
wget https://github.com/libMesh/libmesh/releases/download/v1.6.2/libmesh-1.6.2.tar.gz
tar xvfz libmesh-1.6.2.tar.gz
mv libmesh-1.6.2 LIBMESH

    First, a debug build:

    (Note that if you are using your system MPI, you must modify the following to reflect the correct path to your MPI compilers)

    cd $HOME/sfw/libmesh/1.6.2
mkdir objs-debug
cd objs-debug
../LIBMESH/configure \
    --prefix=$HOME/sfw/libmesh/1.6.2/1.6.2-debug \
    --with-methods=dbg \
    CFLAGS="-I$HOME/sfw/petsc/3.17.5/mac-debug/include" \
    PETSC_DIR=$HOME/sfw/petsc/3.17.5 \
    PETSC_ARCH=mac-debug \
    CC=/usr/local/bin/mpicc \
    CXX=/usr/local/bin/mpicxx \
    FC=/usr/local/bin/mpif90 \
    F77=/usr/local/bin/mpif90 \
    --enable-exodus \
    --enable-triangle \
    --enable-petsc-required \
    --disable-boost \
    --disable-eigen \
    --disable-hdf5 \
    --disable-openmp \
    --disable-perflog \
    --disable-pthreads \
    --disable-tbb \
    --disable-timestamps \
    --disable-reference-counting \
    --disable-strict-lgpl \
    --disable-glibcxx-debugging \
    --disable-vtk \
    --with-thread-model=none
    make -j4
    make -j4 install

    Then, an optimized build:

    (Note that if you are using your system MPI, you must modify the following to reflect the correct path to your MPI compilers)

    cd $HOME/sfw/libmesh/1.6.2
mkdir objs-opt
cd objs-opt
../LIBMESH/configure \
    --prefix=$HOME/sfw/libmesh/1.6.2/1.6.2-opt \
    --with-methods=opt \
    PETSC_DIR=$HOME/sfw/petsc/3.17.5 \
    PETSC_ARCH=mac-opt \
    CFLAGS="-I$HOME/sfw/petsc/3.17.5/mac-opt/include" \
    CC=/usr/local/bin/mpicc \
    CXX=/usr/local/bin/mpicxx \
    FC=/usr/local/bin/mpif90 \
    F77=/usr/local/bin/mpif90 \
    --enable-exodus \
    --enable-triangle \
    --enable-petsc-required \
    --disable-boost \
    --disable-eigen \
    --disable-hdf5 \
    --disable-openmp \
    --disable-perflog \
    --disable-pthreads \
    --disable-tbb \
    --disable-timestamps \
    --disable-reference-counting \
    --disable-strict-lgpl \
    --disable-glibcxx-debugging \
    --disable-vtk \
    --with-thread-model=none
    make -j4
    make -j4 install

    IBAMR

    As of version 0.8.0, IBAMR supports compilation with both the older autotools-based system (i.e., the configure script) and CMake. The CMake version is now the preferred build system since it supports installing IBAMR and also shared object libraries. You can install cmake through homebrew. We first describe how to build IBAMR using CMake. Autotools instructions follow next.

    Compiling IBAMR with CMake

    First, download IBAMR:

    cd $HOME/sfw
mkdir ibamr
cd ibamr
wget https://github.com/IBAMR/IBAMR/archive/v0.15.0.tar.gz
tar xf v0.15.0.tar.gz
mv IBAMR-0.15.0 IBAMR

    Next, configure and compile a debugging build of IBAMR.

    If you are using your system copy of MPI then you can ommit the line containing -DMPI_ROOT: CMake will find it automatically.

    cd $HOME/sfw/ibamr
mkdir ibamr-debug
cd ibamr-debug
cmake \
  -DCMAKE_BUILD_TYPE=Debug \
  -DCMAKE_INSTALL_PREFIX=$HOME/sfw/ibamr/0.15.0/0.15.0-debug \
  -DBOOST_ROOT=/usr/local \
  -DPETSC_ROOT=$HOME/sfw/petsc/3.17.5/mac-debug \
  -DMPI_ROOT=/usr/local \
  -DHYPRE_ROOT=$HOME/sfw/petsc/3.17.5/mac-debug \
  -DSAMRAI_ROOT=$HOME/sfw/samrai/2.4.4/mac-g++-debug \
  -DHDF5_ROOT=/usr/local \
  -DSILO_ROOT=$HOME/sfw/silo/4.11 \
  -DLIBMESH_ROOT=$HOME/sfw/libmesh/1.6.2/1.6.2-debug \
  -DLIBMESH_METHOD=DBG \
  ../IBAMR
make -j4
make -j4 install

    You may now compile and run the IBAMR examples, e.g.,

    cd $HOME/sfw/ibamr/ibamr-debug
make -j4 examples-IB
cd examples/IB/ex0
./main2d input2d

    Next, configure and compile an optimized build of IBAMR. For CMake, you should pass -DCMAKE_CXX_FLAGS="-march=native -O3" and -DCMAKE_Fortran_FLAGS="-march=native -O3" to CMake to use high levels of optimization.

    cd $HOME/sfw/ibamr
mkdir ibamr-opt
cd ibamr-opt
cmake \
  -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_INSTALL_PREFIX=$HOME/sfw/ibamr/0.15.0/0.15.0-opt \
  -DBOOST_ROOT=/usr/local \
  -DPETSC_ROOT=$HOME/sfw/petsc/3.17.5/mac-opt \
  -DMPI_ROOT=/usr/local \
  -DHYPRE_ROOT=$HOME/sfw/petsc/3.17.5/mac-opt \
  -DSAMRAI_ROOT=$HOME/sfw/samrai/2.4.4/mac-g++-opt \
  -DHDF5_ROOT=/usr/local \
  -DSILO_ROOT=$HOME/sfw/silo/4.11 \
  -DLIBMESH_ROOT=$HOME/sfw/libmesh/1.6.2/1.6.2-opt \
  -DLIBMESH_METHOD=OPT \
  ../IBAMR
make -j4
make -j4 install

    You may now compile and run the IBAMR examples with compiler optimizations enabled, e.g.,

    cd $HOME/sfw/ibamr/ibamr-opt
make -j4 examples-IB
cd examples/IB/ex0
./main2d input2d

    With CMake all target names need to be unique: i.e., unlike autotools, you cannot go into a subdirectory and run make examples to compile only a subset of all examples. You have to build all examples (or all examples in a particular group) at once in the top-level build directory.

    Compiling IBAMR with autotools (i.e., using ./configure)

    First, download IBAMR:

    cd $HOME/sfw
mkdir ibamr
cd ibamr
wget https://github.com/IBAMR/IBAMR/archive/v0.15.0.tar.gz
tar xf v0.15.0.tar.gz
mv IBAMR-0.15.0 IBAMR

    IBAMR does not set any compiler flags (with the sole exception of the C++ version flag): instead, the compiler flags are set to the provided values of CFLAGS, CXXFLAGS, and FCFLAGS. The values given below are good defaults for Clang, GCC, and Intel compilers. While optional, we recommend adding some warning suppression flags to silence some unnecessarily pedantic warnings that show up in newer versions of GCC and clang:

    Next, configure and compile a debugging build of IBAMR:

    (Note that if you are using your system MPI, you must modify the following to reflect the correct path to your MPI compilers)

    cd $HOME/sfw/ibamr
mkdir ibamr-objs-dbg
cd ibamr-objs-dbg
export PETSC_ARCH=mac-debug
export PETSC_DIR=$HOME/sfw/petsc/3.17.5
../IBAMR/configure \
CFLAGS="-g -O1 -Wall" \
CXXFLAGS="-g -O1 -Wall" \
FCFLAGS="-g -O1 -Wall" \
CC=/usr/local/bin/mpicc \
CXX=/usr/local/bin/mpicxx \
FC=/usr/local/bin/mpif90 \
CPPFLAGS="-DOMPI_SKIP_MPICXX" \
--with-hypre=$PETSC_DIR/$PETSC_ARCH \
--with-samrai=$HOME/sfw/samrai/2.4.4/mac-g++-debug \
--with-hdf5=/usr/local \
--with-silo=$HOME/sfw/silo/4.11 \
--with-boost=/usr/local \
--enable-libmesh \
--with-libmesh=$HOME/sfw/libmesh/1.6.2/1.6.2-debug \
--with-libmesh-method=dbg
make -j4

    IBAMR’s configure script will automatically add the correct flag to enable C++11 support.

    You may now compile and run the IBAMR examples, e.g.,

    cd $HOME/sfw/ibamr/ibamr-objs-debug/examples/IB/explicit/ex0
make -j4 examples
./main2d input2d

    Next, configure and compile an optimized build of IBAMR. If you are running on a cluster you may want to use additional optimization flags: please refer to the note at the top of the page for more information on how to do this.

    (Note that if you are using your system MPI, you must modify the following to reflect the correct path to your MPI compilers)

    cd $HOME/sfw/ibamr
mkdir ibamr-objs-opt
cd ibamr-objs-opt
export PETSC_ARCH=mac-opt
export PETSC_DIR=$HOME/sfw/petsc/3.17.5
../IBAMR/configure \
  CC=/usr/local/bin/mpicc \
  CXX=/usr/local/bin/mpicxx \
  F77=/usr/local/bin/mpif90 \
  FC=/usr/local/bin/mpif90 \
  MPICC=/usr/local/bin/mpicc \
  MPICXX=/usr/local/bin/mpicxx \
  CFLAGS="-O3 -pipe -Wall" \
  CXXFLAGS="-O3 -pipe -Wall" \
  FCFLAGS="-O3 -pipe -Wall" \
  CPPFLAGS="-DOMPI_SKIP_MPICXX" \
  --with-hypre=$PETSC_DIR/$PETSC_ARCH \
  --with-samrai=$HOME/sfw/samrai/2.4.4/mac-g++-opt \
  --with-hdf5=/usr/local \
  --with-silo=$HOME/sfw/silo/4.11 \
  --with-boost=/usr/local \
  --enable-libmesh \
  --with-libmesh=$HOME/sfw/libmesh/1.6.2/1.6.2-opt \
  --with-libmesh-method=opt
  make -j4

    You may now compile and run the IBAMR examples with compiler optimizations enabled, e.g.,

    cd $HOME/sfw/ibamr/ibamr-objs-opt/examples/IB/explicit/ex0
make -j4 examples
./main2d input2d