INSTALL

Requirements
============
1. DataSpaces requires MPI. 


Quick Installation Instructions
===============================
The following steps are needed to configure, build and install DataSpaces.

1. Configure
# Examples for the configure command on supported architectures and networks. Please customize it according to your specific system configurations and environments (If there is no 'configure' file, please run ./autogen.sh to generate it.).
# Cray XE and XK series
$ ./configure CC=cc FC=ftn
* For Aries based system, e.g. EOS at ORNL, please add:
  --enable-aries

# Infiniband cluster
$ ./configure CC=mpicc FC=mpif90

* For TACC Stampede InfiniBand Cluster
  $ module unload intel
  $ module unload mvapich2/2.1
  $ module load intel/13.0.2.146
  $ module load mvapich2/1.9a2
  $ ./configure CC=mpicc FC=mpif90 CFLAGS="-DHAVE_INFINIBAND -L/opt/ofed/lib64/ -lrdmacm -I/opt/ofed/include/" --with-ib-interface=ib

# IBM BlueGene/P
$ ./configure CC=mpixlc FC=mpixlf90 CFLAGS="-g -O0 -qarch=450 -qtune=450" --with-dcmf="/bgsys/drivers/ppcfloor"

# IBM BlueGene/Q
$ ./configure CC=/bgsys/drivers/ppcfloor/comm/xl/bin/mpixlc_r FC=/bgsys/drivers/ppcfloor/comm/xl/bin/mpixlf90 CFLAGS="-O0 -g -qlanglvl=extc99 -O3 -qarch=qp -qtune=qp -qfullpath"

# TCP/IP
$ ./configure CC=[C_COMPILER_OPTION] FC=[Fortran_COMPILER_OPTION] --enable-dart-tcp
* [C_COMPILER_OPTION]: C compiler in cluster; [Fortran_COMPILER_OPTION]: Fortran compiler in cluster.

2. Some of the useful configure options
 --prefix=/install/path
   Set the directory where DataSpaces will be installed.

 --with-max-num-array-dimension=integer value
   This option is used to configure the maximum number
   of array dimension that can be supported in
   DataSpaces/DIMES. Default value is set as 3. Note:
   the value can not be set larger than 10, or smaller than 3.

 --enable-dimes
   Build DataSpaces with DIstributed MEmory Space (DIMES) support.
   DIMES feature uses local RDMA memory buffers across application processes to build
   a virtual shared data space in-situ, which can be queried and directly accessed by 
   other applications in a data coupling workflow. Currently this feature is only 
   supported on Cray Gemini networks, Infiniband cluster and on IBM BG/P and BG/Q systems.

 --with-dimes-rdma-buffer-size=integer value for megabytes
   This option specifies the maximum amount of RDMA memory buffer that
   can be used by DIMES in each application process. DIMES RDMA memory buffer is used
   to locally cache the data written by dimes_put(), and is also used by
   dimes_get() to fetch data in remote memory buffer. Must be used with
   --enable-dimes option. Default value is set as 64MB.

 --enable-dart-tcp 
   This option will enable DataSpaces to build on ethernet using TCP/IP socket interfaces. 
   Since socket is available in most of the current clusters, this option will avoid the 
   conflict of selections between IP network and other advanced networks (e.g. IB, Gemini). 

 --enable-aries
   With this option, it will get the right shared protection domain information for 
   application access on Aries-based systems, such as EOS system at ORNL. Before adding 
   this option, please run 'apstat -P' to check if your Aries based system support shared 
   protection domain.

 --with-gni-ptag = PTAG DECIMAL VALUE
 --with-gni-cookie = COOKIE HEXA VALUE
   This two options directly give the values of ptag+cookie pair to DataSpaces in uGNI 
   based systems. When DataSpaces is running at Aries-based systems, ptag will be 
   automatically assigned by the system itself. Therefore, in those systems, only 
   --with-gni-cookie is required to provided and activated.

 --with-infiniband-msg-queue-size=integer value
   User can use this option to change the size of the messaging queue on DataSpaces servers.
   If you receive errors stating the messaging queue is full. You should use this option to
   increase the size of the messaging queue.

   For applications using DIMES, dimes_put_sync_all() or dimes_put_sync_group()
   API functions (see dimes_interface.h for detailed documentation) should be
   called periodically to free data objects cached in DIMES. Where there is no
   sufficient memory space, dimes_put()/dimes_get() would fail and return errors. 
   
 --enable-ib-cpp=yes
   To use cpp implementation of dart communication layer using infiniband. If this is enabled, some
   care needs to be taken on the configure command; CXX=mpicxx, LDFLAGS="-lstdc++ -lmpi_cxx -lpthread"
   e.g., 
   ./configure CC=mpicc FC=mpif90 CXX=mpicxx 
     LDFLAGS="-lstdc++ -lmpi_cxx -lpthread" \
     --enable-ib-cpp=yes --with-ib-interface=ib0 --prefix="install_dir"
   
3. Build & Install 
# Under project top directory
$ make
$ make install

For more information on installing DataSpaces and running the examples, or 
if you have any other questions, please visit the following web page:
http://personal.cac.rutgers.edu/TASSL/projects/data/support.html