Third-Party Libraries

These instructions describe building and installing IBAMR and its required third-party libraries on a Linux system. Minimal modifications should be required for similar operating systems. The instructions below create both a debug build, where the libraries are compiled with additional checking and can be used with a debugger, and an optimized build, which is generally several times faster than the debug build because most error checking is turned off and debugging information is not available.

All libraries specified below are compiled with the make program. We recommend running make in parallel whenever possible: make is smart enough to compile multiple files at once. One can specify the number of jobs, or concurrent copies of make to run, with the -j flag: i.e., running make -j4 will compile four files at once. We recommend using the same number of make jobs as physical processors on a machine. The sample instructions here use four jobs.

Picking good optimization flags

IBAMR’s performance in release mode is very sensitive to the choice of optimization flags. For example: switching from -O3 to -march=native -O3 roughly doubled the performance of some IBFE-based codes. This is because a large fraction of the time spent in IBAMR is spent doing structured grid calculations that can benefit heavily from applying processor-specific optimizations.

If you want good performance then we recommend adding -march=native to the flags chosen below for optimization purposes (i.e., any place where we write -O3, we recommend you use -O3 -march=native). This is not listed as the default compiler flag because it generates processor-specific instructions, which may not be valid if you compile on one machine and execute on another (e.g., if you compile on a cluster’s login node and then run on a partition that uses a different set of hardware). To get around this we recommend compiling IBAMR and IBAMR applications in the same place you run them, e.g., on the same partition of a cluster.

Most of the time spent by IBAMR applications is spent inside IBAMR functions, so most of the benefit of processor-specific compilation flags is achieved by compiling IBAMR itself (i.e., it is not necessary to recompile dependencies with -march=native, but it can only help).

For more information see The GCC page on compiler flags. The flag -march=native is supported on all major C++ compilers (GCC, Clang, and Intel).

Setup

cd $HOME
mkdir sfw
cd sfw
mkdir linux

Boost

Currently, IBAMR requires Boost version greater than or equal to 1.57, and only requires the core header-only part of Boost.

NOTE: A subset of Boost 1.60 is bundled with IBAMR and is built automatically by IBAMR configure. It is also possible to use an external installation of Boost with IBAMR.

NOTE: If you configure IBAMR with libMesh and libMesh has been configured to use boost, then both libMesh and IBAMR must use the same external copy of Boost: the copy of Boost included with libMesh does not contain some headers that IBAMR requires.

cd $HOME/sfw/linux
mkdir boost
cd boost
tar xvfz path/to/boost_1_60_0.tar.gz
mv boost_1_60_0 1.60.0
export BOOST_ROOT=$HOME/sfw/linux/boost/1.60.0
mkdir $BOOST_ROOT/include
ln -s $BOOST_ROOT/boost $BOOST_ROOT/include

Notice it is not necessary to build Boost; we only need headers.

Eigen

Currently, IBAMR requires Eigen version greater than or equal to 3.2.5.

NOTE: Eigen 3.2.5 is bundled with IBAMR and is built automatically by IBAMR configure. It is also possible to use an external installation of Eigen with IBAMR. If you will be using libMesh, you must use an external Eigen.

HDF5

Newer versions of HDF5 (1.12.0 and newer) include API changes that are not compatible with Silo, so we recommend using a recent version of HDF5 but not newer than 1.10.6 unless you know what you are doing. We recommend either using the copy of HDF5 already installed on your computer or letting PETSc build it (via the flag --download-hdf5=1). If you let PETSc build HDF5 then all paths supplied later on to indicate where HDF5 is must point to PETSc’s installation directory. However, if you need to compile it yourself, here are the instructions:

cd $HOME/sfw/linux

mkdir hdf5
cd hdf5
tar xvjf path/to/hdf5-1.10.6.gz.bz2
cd hdf5-1.10.6
./configure \
  CC=gcc \
  CXX=g++ \
  FC=gfortran \
  F77=gfortran \
  --enable-build-mode=production \
  --prefix=$HOME/sfw/linux/hdf5/1.10.6
make -j4
make -j4 check
make -j4 install

Silo

(optional)

We recommend using the latest available version of Silo (currently 4.10), but earlier versions of Silo are also likely to work with IBAMR. We recommend letting PETSc download Silo (via --download-silo=1). If you let PETSc build Silo then all paths supplied later on to indicate where Silo is must point to PETSc’s installation directory.

NOTE: Silo 4.10 requires an installation of zlib that includes its headers and development files, which is not necessarily the same as the standard version of zlib. It is available on Ubuntu systems through apt under the name zlib1g-dev. If you build it from source from http://zlib.net/, specify its location with:

--with-zlib="path/to/zlib/1.2.8/include,path/to/zlib/1.2.8/lib" .

Notice that you have to specify its include and lib directories.

cd $HOME/sfw/linux
tar xvfz path/to/silo-4.10.tar.gz
cd silo-4.10
./configure \
  CC=gcc \
  CXX=g++ \
  FC=gfortran \
  F77=gfortran \
  --prefix=$HOME/sfw/linux/silo/4.10 \
  --disable-silex
make -j4
make -j4 install

Some operating systems, such as Ubuntu 16.04, require that configure be provided with the path to the SZ library via --with-szlib=/usr/lib (or wherever libsz.so is installed).

If you do not have the GNU readline library installed, you may need --without-readline.

muParser

NOTE: A patched version of muParser 2.2.3 is bundled with IBAMR and installed automatically by IBAMR configure. It is also possible to use an external installation of muParser with IBAMR.

MPI

IBAMR requires that MPI be available but does not require the use of a specific MPI library; most of the IBAMR developers use OpenMPI. We recommend usage of a system-provided MPI installation if available.

It is also possible to have PETSc download and install an MPI implementation instead of building it yourself by configuring PETSc with the --download-mpich=1 (where MPICH is an MPI implementation) flag. If use your system MPI, then skip this step and wherever you see the path to the OpenMPI compilers specified, replace it with the path to your system’s MPI compilers (usually mpicc, mpicxx, and mpifort).

If you do not have a system MPI installation available, we recommend using the most recent version of OpenMPI. After downloading the tarball from the OpenMPI website, you may do the following to build it.

cd $HOME/sfw/linux
tar xvfj path/to/openmpi-4.0.2.tar.bz2

cd openmpi-4.0.2
./configure \
  CC=gcc \
  CXX=g++ \
  FC=gfortran \
  F77=gfortran \
  --prefix=$HOME/sfw/linux/openmpi/4.0.2 \
  --enable-orterun-prefix-by-default
make -j4
make -j4 check
make -j4 install

NOTE: You may wish to add the lines:

export PATH=$HOME/sfw/linux/openmpi/4.0.2/bin:$PATH
export MANPATH=$HOME/sfw/linux/openmpi/4.0.2/share/man:$MANPATH

to your ~/.bashrc file.

Hypre

IBAMR requires Hypre version 2.10b or later. We recommend you let PETSc install hypre.

PETSc

IBAMR requires the PETSc (portable toolkit for scientific computing) library. IBAMR is compatible with PETSc versions 3.7 and newer. If you use a new version of PETSc then you will also need to use a new version of libMesh. These instructions are for PETSc version 3.13.4.

These configure options direct PETSc to download and install hypre. First, unpack PETSc:

cd $HOME/sfw
mkdir petsc
cd petsc
wget http://ftp.mcs.anl.gov/pub/petsc/release-snapshots/petsc-3.13.4.tar.gz
tar xvfz petsc-3.13.4.tar.gz
mv petsc-3.13.4 3.13.4
cd 3.13.4

Next, configure and compile a debugging build of PETSc:

(Note that the LDFLAGS option is only necessary with OpenMPI, not MPICH or other MPI flavors.)

(Note that if you are using your system MPI, you must modify the following to reflect the correct path to your MPI compilers)

export PETSC_DIR=$PWD
export PETSC_ARCH=linux-debug
./configure \
  --CC=$HOME/sfw/linux/openmpi/4.0.2/bin/mpicc \
  --CXX=$HOME/sfw/linux/openmpi/4.0.2/bin/mpicxx \
  --FC=$HOME/sfw/linux/openmpi/4.0.2/bin/mpif90 \
  --with-debugging=1 \
  --download-hypre=1 \
  --with-x=0
make -j4
make -j4 test

Finally, configure and compile an optimized build of PETSc: (Note that the LDFLAGS option is only necessary with OpenMPI, not MPICH or other MPI flavors.)

(Note that if you are using your system MPI, you must modify the following to reflect the correct path to your MPI compilers)

export PETSC_DIR=$PWD
export PETSC_ARCH=linux-opt
./configure \
  --CC=$HOME/sfw/linux/openmpi/4.0.2/bin/mpicc \
  --CXX=$HOME/sfw/linux/openmpi/4.0.2/bin/mpicxx \
  --FC=$HOME/sfw/linux/openmpi/4.0.2/bin/mpif90 \
  --COPTFLAGS="-O3" \
  --CXXOPTFLAGS="-O3" \
  --FOPTFLAGS="-O3" \
  --PETSC_ARCH=$PETSC_ARCH \
  --with-debugging=0 \
  --download-hypre=1 \
  --with-x=0
make -j4
make -j4 test

NOTE: You may wish to add the lines:

export PETSC_DIR=$HOME/sfw/petsc/3.13.4
export PETSC_ARCH=linux-opt

to your ~/.bashrc file.

SAMRAI

IBAMR requires SAMRAI version 2.4.4 and does not work with newer (or older) versions of SAMRAI.

First, download SAMRAI (shown here via wget) and unpack it:

cd $HOME/sfw
mkdir samrai
cd samrai
mkdir 2.4.4
cd 2.4.4
wget https://github.com/LLNL/SAMRAI/archive/refs/tags/2.4.4.tar.gz
tar xf 2.4.4.tar.gz

Next, patch SAMRAI using the patch file ibamr-samrai-fixes.patch:

cd SAMRAI-2.4.4
wget https://github.com/IBAMR/IBAMR/releases/download/v0.10.1/samrai-2.4.4-patch-ibamr-0.10.patch
patch -p1 < samrai-2.4.4-patch-ibamr-0.10.patch

Next, configure and compile a debugging build of SAMRAI. It may be necessary, if you are not using your system installation of MPI, to modify the paths to the MPI compiler wrappers:

cd $HOME/sfw/samrai/2.4.4
mkdir objs-debug
cd objs-debug

../SAMRAI-2.4.4/configure \
  CFLAGS="-fPIC" \
  CXXFLAGS="-fPIC" \
  FFLAGS="-fPIC" \
  --prefix=$HOME/sfw/samrai/2.4.4/linux-g++-debug \
  --with-CC=$HOME/sfw/linux/openmpi/4.0.2/bin/mpicc \
  --with-CXX=$HOME/sfw/linux/openmpi/4.0.2/bin/mpicxx \
  --with-F77=$HOME/sfw/linux/openmpi/4.0.2/bin/mpif90 \
  --with-hdf5=$HOME/sfw/linux/hdf5/1.10.6 \
  --without-petsc \
  --without-hypre \
  --with-silo=$HOME/sfw/linux/silo/4.10 \
  --without-blaslapack \
  --without-cubes \
  --without-eleven \
  --without-kinsol \
  --without-petsc \
  --without-sundials \
  --without-x \
  --with-doxygen \
  --with-dot \
  --enable-debug \
  --disable-opt \
  --enable-implicit-template-instantiation \
  --disable-deprecated
make -j4
make -j4 install

Finally, configure and compile an optimized build of SAMRAI:

cd $HOME/sfw/samrai/2.4.4
mkdir objs-opt
cd objs-opt

../SAMRAI-2.4.4/configure \
  CFLAGS="-O3 -fPIC" \
  CXXFLAGS="-O3 -fPIC" \
  FFLAGS="-O3 -fPIC" \
  --prefix=$HOME/sfw/samrai/2.4.4/linux-g++-opt \
  --with-CC=$HOME/sfw/linux/openmpi/4.0.2/bin/mpicc \
  --with-CXX=$HOME/sfw/linux/openmpi/4.0.2/bin/mpicxx \
  --with-F77=$HOME/sfw/linux/openmpi/4.0.2/bin/mpif90 \
  --with-hdf5=$HOME/sfw/linux/hdf5/1.10.6 \
  --without-hypre \
  --with-silo=$HOME/sfw/linux/silo/4.10 \
  --without-blaslapack \
  --without-cubes \
  --without-eleven \
  --without-kinsol \
  --without-petsc \
  --without-sundials \
  --without-x \
  --with-doxygen \
  --with-dot \
  --disable-debug \
  --enable-opt \
  --enable-implicit-template-instantiation \
  --disable-deprecated
make -j4
make -j4 install

Note that, in both cases, we need to add the -fPIC flag so that SAMRAI is compatible with shared object libraries. Strictly speaking, you don’t need this flag if you are using the autotools-based build system for IBAMR, but you do if you use CMake.

libMesh

libMesh is a finite element library required by the IBFE subsystem. It is not a required dependency of IBAMR but is strongly recommended.

IBAMR is compatible with libMesh 1.1.0 or newer. The instructions below use the latest version of libMesh (1.6.2). If you use libMesh with boost, then you MUST use an external build of boost for both IBAMR and libMesh since the subset of boost included with libMesh does not contain everything that IBAMR needs. We recommend compiling libMesh without boost to avoid this issue.

We recommend compiling libMesh with ExodusII. ExodusII support is enabled by the --enable-exodus flag, which is included below. Many IBAMR examples use libMesh’s ExodusII writer to output data for visualization. The examples can be run without ExodusII but will need to be modified to generate output in another format.

NOTE: libMesh assumes that PETSc was installed in place, i.e., without calling make install at the end. The instructions on this page assume that PETSc is installed in place. However, if you choose instead to run make install, then one should provide the full path to the PETSc installation as PETSC_DIR: e.g., in this case, one would provide PETSC_DIR=$HOME/sfw/petsc/3.13.4/linux-debug as an argument.

cd $HOME/sfw/linux
mkdir libmesh
cd libmesh
mkdir 1.6.2
cd 1.6.2
wget https://github.com/libMesh/libmesh/releases/download/v1.6.2/libmesh-1.6.2.tar.gz
tar xvfz libmesh-1.6.2.tar.gz
mv libmesh-1.6.2 LIBMESH

First, a debug build:

(Note that if you are using your system MPI, you must modify the following to reflect the correct path to your MPI compilers)

cd $HOME/sfw/linux/libmesh/1.6.2
mkdir objs-debug
cd objs-debug
../LIBMESH/configure \
    --prefix=$HOME/sfw/linux/libmesh/1.6.2/1.6.2-debug \
    --with-methods=dbg \
    PETSC_DIR=$HOME/sfw/petsc/3.13.4 \
    PETSC_ARCH=linux-debug \
    CC=$HOME/sfw/linux/openmpi/4.0.2/bin/mpicc \
    CXX=$HOME/sfw/linux/openmpi/4.0.2/bin/mpicxx \
    FC=$HOME/sfw/linux/openmpi/4.0.2/bin/mpif90 \
    F77=$HOME/sfw/linux/openmpi/4.0.2/bin/mpif90 \
    --enable-exodus \
    --enable-triangle \
    --enable-petsc-required \
    --disable-boost \
    --disable-eigen \
    --disable-hdf5 \
    --disable-openmp \
    --disable-perflog \
    --disable-pthreads \
    --disable-tbb \
    --disable-timestamps \
    --disable-reference-counting \
    --disable-strict-lgpl \
    --disable-glibcxx-debugging \
    --disable-vtk \
    --with-thread-model=none
make -j4
make -j4 install

Then, an optimized build:

(Note that if you are using your system MPI, you must modify the following to reflect the correct path to your MPI compilers)

cd $HOME/sfw/linux/libmesh/1.6.2
mkdir objs-opt
cd objs-opt
../LIBMESH/configure \
    --prefix=$HOME/sfw/linux/libmesh/1.6.2/1.6.2-opt \
    --with-methods=opt \
    PETSC_DIR=$HOME/sfw/petsc/3.13.4 \
    PETSC_ARCH=linux-opt \
    CC=$HOME/sfw/linux/openmpi/4.0.2/bin/mpicc \
    CXX=$HOME/sfw/linux/openmpi/4.0.2/bin/mpicxx \
    FC=$HOME/sfw/linux/openmpi/4.0.2/bin/mpif90 \
    F77=$HOME/sfw/linux/openmpi/4.0.2/bin/mpif90 \
    --enable-exodus \
    --enable-triangle \
    --enable-petsc-required \
    --disable-boost \
    --disable-eigen \
    --disable-hdf5 \
    --disable-openmp \
    --disable-perflog \
    --disable-pthreads \
    --disable-tbb \
    --disable-timestamps \
    --disable-reference-counting \
    --disable-strict-lgpl \
    --disable-glibcxx-debugging \
    --disable-vtk \
    --with-thread-model=none
make -j4
make -j4 install

IBAMR


As of version 0.8.0, IBAMR supports compilation with both the older autotools-based system (i.e., the configure script) and CMake. The CMake version is now the preferred build system since it supports installing IBAMR and also shared object libraries.

Compiling IBAMR with CMake

First, download IBAMR:

cd $HOME/sfw
mkdir ibamr
cd ibamr
wget https://github.com/IBAMR/IBAMR/archive/v0.10.1.tar.gz
tar xf v0.10.1.tar.gz
mv IBAMR-0.10.1 IBAMR

Next, configure and compile a debugging build of IBAMR.

If you are using your system copy of MPI then you can ommit the line containing -DMPI_ROOT: CMake will find it automatically.

cd $HOME/sfw/ibamr
mkdir ibamr-debug
cd ibamr-debug
cmake \
  -DCMAKE_BUILD_TYPE=Debug \
  -DCMAKE_INSTALL_PREFIX=$HOME/sfw/linux/ibamr/0.10.1/0.10.1-debug \
  -DBOOST_ROOT=$HOME/sfw/linux/boost/1.60.0 \
  -DPETSC_ROOT=$HOME/sfw/petsc/3.13.4/linux-debug \
  -DMPI_ROOT=$HOME/sfw/linux/openmpi/4.0.2 \
  -DHYPRE_ROOT=$HOME/sfw/petsc/3.13.4/linux-debug \
  -DSAMRAI_ROOT=$HOME/sfw/samrai/2.4.4/linux-g++-debug \
  -DHDF5_ROOT=$HOME/sfw/linux/hdf5/1.10.6 \
  -DSILO_ROOT=$HOME/sfw/linux/silo/4.10 \
  -DLIBMESH_ROOT=$HOME/sfw/linux/libmesh/1.6.2/1.6.2-debug \
  -DLIBMESH_METHOD=DBG \
  ../IBAMR
make -j4
make -j4 install

If you had to install your own copy of zlib for Silo, you will also need to specify the installation directory of zlib here via -DZLIB_ROOT=PATH, where PATH is the directory where zlib was installed.

You may now compile and run the IBAMR examples, e.g.,

cd $HOME/sfw/ibamr/ibamr-debug
make -j4 examples-IB
cd examples/IB/ex0
./main2d input2d

Next, configure and compile an optimized build of IBAMR. If you are running on a cluster you may want to use additional optimization flags: please refer to the note at the top of the page for more information on how to do this. For CMake, you should pass -DCMAKE_CXX_FLAGS="-march=native -O3" and -DCMAKE_Fortran_FLAGS="-march=native -O3" to CMake to use high levels of optimization.

If you are using your system copy of MPI then you can ommit the line containing -DMPI_ROOT: CMake will find it automatically.

cd $HOME/sfw/ibamr
mkdir ibamr-opt
cd ibamr-opt
cmake \
  -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_INSTALL_PREFIX=$HOME/sfw/linux/ibamr/0.10.1/0.10.1-opt \
  -DBOOST_ROOT=$HOME/sfw/linux/boost/1.60.0 \
  -DPETSC_ROOT=$HOME/sfw/petsc/3.13.4/linux-opt \
  -DMPI_ROOT=$HOME/sfw/linux/openmpi/4.0.2 \
  -DHYPRE_ROOT=$HOME/sfw/petsc/3.13.4/linux-opt \
  -DSAMRAI_ROOT=$HOME/sfw/samrai/2.4.4/linux-g++-opt \
  -DHDF5_ROOT=$HOME/sfw/linux/hdf5/1.10.6 \
  -DSILO_ROOT=$HOME/sfw/linux/silo/4.10 \
  -DLIBMESH_ROOT=$HOME/sfw/linux/libmesh/1.6.2/1.6.2-opt \
  -DLIBMESH_METHOD=OPT \
  ../IBAMR
make -j4
make -j4 install

If you had to install your own copy of zlib for Silo, you will also need to specify the installation directory of zlib here via -DZLIB_ROOT=PATH, where PATH is the directory where zlib was installed.

You may now compile and run the IBAMR examples with compiler optimizations enabled, e.g.,

cd $HOME/sfw/ibamr/ibamr-opt
make -j4 examples-IB
cd examples/IB/ex0
./main2d input2d

Compiling IBAMR with ./configure

First, download IBAMR:

cd $HOME/sfw
mkdir ibamr
cd ibamr
wget https://github.com/IBAMR/IBAMR/archive/v0.10.1.tar.gz
tar xf v0.10.1.tar.gz
mv IBAMR-0.10.1 IBAMR

IBAMR does not set any compiler flags (with the sole exception of the C++ version flag): instead, the compiler flags are set to the provided values of CFLAGS, CXXFLAGS, and FCFLAGS. The values given below are good defaults for Clang, GCC, and Intel compilers. While optional, we recommend adding some warning suppression flags to silence some unnecessarily pedantic warnings that show up in newer versions of GCC and clang:

Next, configure and compile a debugging build of IBAMR:

(Note that if you are using your system MPI, you must modify the following to reflect the correct path to your MPI compilers)

cd $HOME/sfw/ibamr
mkdir ibamr-objs-dbg
cd ibamr-objs-dbg
export BOOST_ROOT=$HOME/sfw/linux/boost/1.60.0
export PETSC_ARCH=linux-debug
export PETSC_DIR=$HOME/sfw/petsc/3.13.4
../IBAMR/configure \
  CFLAGS="-g -O1 -Wall" \
  CXXFLAGS="-g -O1 -Wall" \
  FCFLAGS="-g -O1 -Wall" \
  CC=$HOME/sfw/linux/openmpi/4.0.2/bin/mpicc \
  CXX=$HOME/sfw/linux/openmpi/4.0.2/bin/mpicxx \
  FC=$HOME/sfw/linux/openmpi/4.0.2/bin/mpif90 \
  CPPFLAGS="-DOMPI_SKIP_MPICXX" \
  --with-hypre=$PETSC_DIR/$PETSC_ARCH \
  --with-samrai=$HOME/sfw/samrai/2.4.4/linux-g++-debug \
  --with-hdf5=$HOME/sfw/linux/hdf5/1.10.6 \
  --with-silo=$HOME/sfw/linux/silo/4.10 \
  --with-boost=$HOME/sfw/linux/boost/1.60.0 \
  --enable-libmesh \
  --with-libmesh=$HOME/sfw/linux/libmesh/1.6.2/1.6.2-debug \
  --with-libmesh-method=dbg
make -j4

If you had to install your own copy of zlib for Silo, you will also need to specify the installation directory of zlib here via --with-zlib=PATH, where PATH is the directory where zlib was installed.

IBAMR’s configure script will automatically add the correct flag to enable C++11 support.

You may now compile and run the IBAMR examples, e.g.,

cd $HOME/sfw/ibamr/ibamr-objs-debug/examples/IB/explicit/ex0
make -j4 examples
./main2d input2d

Next, configure and compile an optimized build of IBAMR. If you are running on a cluster you may want to use additional optimization flags: please refer to the note at the top of the page for more information on how to do this.

(Note that if you are using your system MPI, you must modify the following to reflect the correct path to your MPI compilers)

cd $HOME/sfw/ibamr
mkdir ibamr-objs-opt
cd ibamr-objs-opt
export PETSC_ARCH=linux-opt
export PETSC_DIR=$HOME/sfw/petsc/3.13.4
../IBAMR/configure \
  CC=$HOME/sfw/linux/openmpi/4.0.2/bin/mpicc \
  CXX=$HOME/sfw/linux/openmpi/4.0.2/bin/mpicxx \
  F77=$HOME/sfw/linux/openmpi/4.0.2/bin/mpif90 \
  FC=$HOME/sfw/linux/openmpi/4.0.2/bin/mpif90 \
  MPICC=$HOME/sfw/linux/openmpi/4.0.2/bin/mpicc \
  MPICXX=$HOME/sfw/linux/openmpi/4.0.2/bin/mpicxx \
  CFLAGS="-O3 -pipe -Wall" \
  CXXFLAGS="-O3 -pipe -Wall" \
  FCFLAGS="-O3 -pipe -Wall" \
  CPPFLAGS="-DOMPI_SKIP_MPICXX" \
  --with-hypre=$PETSC_DIR/$PETSC_ARCH \
  --with-samrai=$HOME/sfw/samrai/2.4.4/linux-g++-opt \
  --with-hdf5=$HOME/sfw/linux/hdf5/1.10.6 \
  --with-silo=$HOME/sfw/linux/silo/4.10 \
  --with-boost=$HOME/sfw/linux/boost/1.60.0 \
  --enable-libmesh \
  --with-libmesh=$HOME/sfw/linux/libmesh/1.6.2/1.6.2-opt \
  --with-libmesh-method=opt
make -j4

You may now compile and run the IBAMR examples with compiler optimizations enabled, e.g.,

cd $HOME/sfw/ibamr/ibamr-objs-opt/examples/IB/explicit/ex0
make -j4 examples
./main2d input2d