GravitySetup

This page details what you need to set up in Linux and Windows so that you can build Gravity for C++ and Java.

Note on current and future build plans

We are in the process of moving to cmake for all our builds, but currently only the Windows cmake build has been fully tested. The cmake process is described in the Windows section below, but we still use the autoconf approach as the official build process on Linux. Soon we will fully convert builds on all platforms to be cmake only.

If interested however the Linux cmake instructions are provided below as well.

Linux

The Linux setup is fairly straightforward. You just need to install:

Oracle's Java JDK
swig
maven
bison/flex
libboost-dev

The last four items above can probably be installed using the package installation system on your flavor of Linux (e.g. sudo apt-get install swig). If a newer version of swig is needed than is provided by your package management system, you can download the needed version from http://www.swig.org/ and build it.

Installing Java may be a little more involved. You can skip building Java with the --without-java option to the configure script.
You can download an installer here: http://www.oracle.com/technetwork/java/javase/downloads/index.html. The complete installation depends on your system. Some info on installing on Ubuntu can be found here: http://askubuntu.com/questions/67909/how-do-i-install-oracle-jdk-6 or here: http://www.wikihow.com/Install-Oracle-Java-JDK-on-Ubuntu-Linux.

Python Support
For Python support, there are a couple of additional build dependencies (note that you can disable Python support with the --without-python option to the configure script):

python-dev
- This can be installed with your package management system
- Note: The Python version used here must match the runtime version of Python. This is true down to the subversion - e.g. Python 3.7 must be built against to run with a Python 3.7 runtime. Python 3.6 may not work in this case.
python-protobuf
- This can be installed with your package management system or pip (e.g. sudo pip install protobuf)

You must also download and build the following:

ZeroMQ http://download.zeromq.org/zeromq-3.2.3.tar.gz

Extract in ~/gravity_deps/ Run ./configure, make,and make install
Google Protocol Buffers https://github.com/google/protobuf/releases/download/v2.6.1/protobuf-2.6.1.tar.gz

Extract in ~/gravity_deps/ Run ./configure, make, and make install
sudo ldconfig

For Java: Build the Java runtime library using Maven (see the readme in java). NOTE: Maven does not use the standard proxy environment variables (http_proxy, etc) - you must configure Maven explicitly. Directions for this can be found here: https://maven.apache.org/guides/mini/guide-proxies.html.

For Python: Also follow these steps:
- cd to the python directory in your protobuf distribution
- Install pip if it is not already available (e.g. sudo apt-get install python-pip)
- python setup.py build
Copy dependencies into gravity/deps
- cp [HOME]/gravity_deps/protobuf-2.6.1/src/.libs/libproto* [HOME]/gravity/deps
- cp [HOME]/gravity_deps/zeromq-3.2.3/src/.libs/libzmq.* [HOME]/gravity/deps
- cp [HOME]/gravity_deps/zeromq-3.2.3/include/* [HOME]/gravity/deps
- cp [HOME]/gravity_deps/protobuf-2.6.1/src/protoc [HOME]/gravity/deps
- cp [HOME]/gravity_deps/protobuf-2.6.1/java/target/protobuf-2.6.1.jar [HOME]/gravity/deps/protobuf-java.jar
- cp -r [HOME]/gravity_deps/protobuf-2.6.1/python/google [HOME]/gravity/deps
Clone from the git repository (or otherwise acquire) the Gravity source tree
cd to the gravity source directory (e.g "\home\gravity")

./configure --with-protobuf-libdir=[HOME]/gravity/deps --with-protobuf-incdir=[HOME]/gravity/deps --with-zeromq-libdir=[HOME]/gravity/deps --with-zeromq-incdir=[HOME]/gravity/deps --with-protoc=[HOME]/gravity/deps/protoc JAVAPROTOBUF_DIR=[HOME]/gravity/deps/protobuf-java.jar CPPFLAGS=-std=c++11 GUAVAJAR_DIR=[HOME]/gravity/src/api/MATLAB/guava-13.0.1.jar

[!NOTE] If building against Python3, additional arguments to the configure script are required - you must explicitly reference the Python3 build environment using the PYTHON, PYTHON_CONFIG and PYTHON_INCLUDE parameters.

For example, you would add these parameters to the configure command above (updated to your machine's Python3 paths):
PYTHON=python3 PYTHON_CONFIG=/usr/local/lib/python3.8/dist-packages/config.py PYTHON_INCLUDE=-I/usr/include/python3.8

make

make test-prep

make test

make distributable
If needed for certain project, create or edit /etc/ld.so.conf.d/gravity.conf and add the following lines
- /[HOME]/gravity/deps
- /[HOME]/gravity/lib
- /[HOME]/gravity/src/api/bin
- /[HOME]/gravity/src/keyvalue_parser
sudo ldconfig
Additional Python Runtime Dependencies
- setuptools (https://pypi.org/project/setuptools/)
- packaging (https://pypi.org/project/packaging/)
- pyparsing (https://pypi.org/project/pyparsing/)
- six (https://pypi.org/project/six/)
Each of these can be installed via pip or (probably) by your package management system (e.g. sudo apt-get install python-setuptools python-packaging python-pyparsing python-six).

Linux cmake instructions

This was tested on a fresh Ubuntu 18 VM, after just installing cmake 3.22 from cmake.org. The '$' represents a command prompt.

$ sudo apt-get install default-jdk python python-dev python-setuptools bison flex swig build-essential
$ cd ~/git/gravity/
$ mkdir build && cd build
$ cmake -DSKIP_PYTHON=OFF -DSKIP_JAVA=OFF -DCMAKE_INSTALL_PREFIX=`pwd`/install ..
$ cmake --build . –-target install
$ export LD_LIBRARY_PATH=~/git/gravity/build/install/lib:$LD_LIBRARY_PATH
$ ~/git/gravity/build/install/bin/ServiceDirectory &

To run the tests: Depending on whether it was an 'external' build or not, either run:

$ ctest --verbose
OR
$ ctest --test-dir gravity_external_examples_tests-prefix/src/gravity_external_examples_tests-build –-verbose

Windows 64 bit build

The tools below are required to build the 64 bit Gravity libraries.

Java JDK if not skipping the Java wrapper https://www.oracle.com/java/technologies/javase-downloads.html
Python >= 2.7.9 plus pip if not skipping the python wrapper

Building Gravity

Clone from the git repository (or otherwise acquire) the Gravity source tree
Install CMake 3.17 or higher https://cmake.org/download/
Run CMake
- Open a command prompt
- set http_proxy and https_proxy as necessary to access external sites
- cd to the gravity repository
- mkdir build && cd build
- Optional CMake settings
  - -DJAVA_HOME= can be set to specify which JDK to use
  - -DSKIP_JAVA=ON to skip building with Java
  - -DSKIP_PYTHON=ON to skip building with Python
  - -DBUILD_EXAMPLES=[ON | OFF] to build the examples, defaults to ON
  - -DProtobuf_SRC_ROOT_FOLDER=<path to your local protobuf source> to use a local protobuf build. Note: the version of Visual Studio you build Gravity with must match the version protobufs is built with. If not specified protobuf will be downloaded externally and built. If you would like to use a protobuf version before their CMake build use this option
  - -DProtobuf_USE_STATIC_LIBS=ON if setting Protobuf_SRC_ROOT_FOLDER this value must be set to ON as well as the protobuf build must be a static library (which is the default in the vsprojects folder)
  - -DZMQ_HOME=<base path to your ZeroMQ binary package>, you can also set the environment variable ZMQ_HOME. If not specified zeromq will be downloaded externally and built. If you would like to use a ZeroMQ version before their CMake build use this option
  - -DGRAVITY_USE_EXTERNAL_PROTOBUF=[ON | OFF] to always use an externally downloaded and built protobuf build regardless of any other settings
  - -DGRAVITY_USE_EXTERNAL_ZEROMQ=[ON | OFF] to always use an externally downloaded and built ZeroMQ build regardless of any other settings
  - -DCMAKE_INSTALL_PREFIX="<installation path>" this is where the build will place the gravity and third party build artifacts. The build will create bin, lib, etc. directories in the specified installation directory. Note that the quotes are needed by some shells to ensure that the path remains intact.
  - If you would like to change the download URL of any packages for example, if using protobuf or ZeroMQ external builds and you would like to change the version, edit cmake/GravityExternalUrls.cmake. See the notes about version requirements in that file.
- For Building With Visual Studio 2022
  - cmake -DCMAKE_INSTALL_PREFIX="../install" -G"Visual Studio 17 2022" -A x64 [ Any options listed above ] .. (Note: the .. is required)
- For Building With Visual Studio 2017
  - cmake -DCMAKE_INSTALL_PREFIX="../install" -G"Visual Studio 15 2017 Win64" [ Any options listed above ] .. (Note: the .. is required)
- cmake --build . --config [debug | release]
- To work in the generated gravity.sln open gravity_external-prefix/src/gravity_external-build/gravity.sln
- Add the binary directories to your path
  - set GRAVITY_HOME=<gravity source dir>/install
  - set Path=%Path%;%GRAVITY_HOME%\bin;%GRAVITY_HOME%\deps\libzmq\bin;%GRAVITY_HOME%\deps\protobuf\bin;%GRAVITY_HOME%\deps\pthreads-w32\bin
  - If you used an local ZeroMQ add its bin directory to your path. If you used a local dynamically linked protobufs, add the dll directory to your path.
- If you used a local protobuf build and built with python:
  - cd <protobuf source dir>/python
  - python.exe setup.py build
  - To run 11-PythonPubSub example, add the following to your PYTHONPATH:
    - <protobuf source dir>/python/build/lib
    - <gravity source dir>/install/lib
    - <gravity source dir>/test/examples/protobuf
  - cd test/examples/11-PythonPubSub
  - start ServiceDirectory.exe
  - python.exe pypub.py

Note on using Python 3.8+ on Windows with Gravity

An additional layer of security was added in Python 3.8 that changes the way library dependencies are managed on Windows (more info here: https://docs.python.org/3.8/whatsnew/3.8.html#bpo-36085-whatsnew). Basically instead of updating the PATH env variable, you need to use os.add_dll_directory to add the needed paths, e.g.:

os.add_dll_directory("C:\\GravityV2\\Test\\Installs\\bin")
os.add_dll_directory("C:\\GravityV2\\Test\\Installs\\deps\\libzmq\\bin")
os.add_dll_directory("C:\\GravityV2\\Test\\Installs\\deps\\pthreads-w32\\bin")

Using Gravity in MATLAB

Linux

MATLAB's Java version

Recent versions of MATLAB ship with Java JDK 1.8. Gravity should be built with a compatible Java version. If you must build Gravity with an incompatible version of Java you will need to configure MATLAB to use that same version. This has the potentially to cause stability issues with MATLAB so proceed with caution. In order to configure MATLAB's java, set a MATLAB_JAVA environment variable to point to your jre directory.

Configure MATLAB's static java class path to include Gravity

Gravity has been tested to work with recent versions of MATLAB (2020a - 2023a). MATLAB version as old as 2010b have been successfully tested but may require slightly different configurations than specified here.

Configure MATLAB's Java classpath

With MATLAB R2023a, for example, add the following lines to /.matlab/R2023a/javaclasspath.txt (where is your user's home directory):

<GRAVITY_HOME>/lib/gravity.jar
<GRAVITY_HOME>/lib/MATLAB/guava-13.0.1.jar
<GRAVITY_HOME>/lib/MATLAB/MATLABGravitySubscriber.jar
<GRAVITY_HOME>/lib/MATLAB/MATLABGravityNode.jar
<GRAVITY_HOME>/deps/protobuf/lib/protobuf-java.jar

where <GRAVITY_HOME> is wherever you installed your Gravity distribution.

Configure MATLAB's library path

Add the following line to the <MATLAB_HOME>/toolbox/local/librarypath.txt (where <MATLAB_HOME> is the MATLAB installation directory):

<GRAVITY_HOME>/lib
<GRAVITY_HOME>/deps

<GRAVITY_HOME>/bin
<GRAVITY_HOME>/deps/libzmq/bin
<GRAVITY_HOME>/deps/protobuf/bin
<GRAVITY_HOME>/deps/pthreads-w32/bin
<GRAVITY_HOME>/deps/spdlog/bin

where <GRAVITY_HOME> is wherever you installed your Gravity distribution.

For some versions of MATLAB, it may be necessary to configure MATLAB to use the system gcc libs

MATLAB ships with its own set of runtime GCC libraries. For compatibility with Gravity we'll want MATLAB to use the system libraries consistent with Gravity. In order to do this, we just need to move the MATLAB libs so that MATLAB will use the system libs. For example:

> cd /usr/local/MATLAB/R2010b/sys/os/glnxa64
> sudo mkdir old
> sudo mv libstdc++.* libgcc_s* old

Set system library path to include Gravity

Create this file (you'll need root privileges): /etc/ld.so.conf.d/gravity.conf Edit that file to include lines for the path to the Gravity lib and deps directory:

<GRAVITY_HOME>/bin
<GRAVITY_HOME>/deps/libzmq/bin
<GRAVITY_HOME>/deps/protobuf/bin
<GRAVITY_HOME>/deps/pthreads-w32/bin
<GRAVITY_HOME>/deps/spdlog/bin

Then, to refresh the system libs:

> sudo ldconfig

Windows

Configure MATLAB's static java class path to include Gravity

Create or edit "javaclasspath.txt" inside the C:\Users<USER>\AppData\Roaming\MathWorks\MATLAB<MATLAB_VERSION> directory. Add the following lines:

<GRAVITY_HOME>\lib\gravity.jar
<GRAVITY_HOME>\deps\protobuf\lib\protobuf-java.jar
<GRAVITY_HOME>\lib\MATLAB\guava-13.0.1.jar
<GRAVITY_HOME>\lib\MATLAB\MATLABGravitySubscriber.jar
<GRAVITY_HOME>\lib\MATLAB\MATLABGravityNode.jar

Configure MATLAB's library path

Edit the "librarypath.txt" file located in C:\Program Files\MATLAB<MATLAB_VERSION>\toolbox\local directory. Add the following lines:

<GRAVITY_HOME>\bin
<GRAVITY_HOME>\deps\libzmq\bin
<GRAVITY_HOME>\deps\protobuf\bin
<GRAVITY_HOME>\deps\pthreads-w32\bin
<GRAVITY_HOME>\deps\spdlog\bin

Configure system path

Add the five paths specified above to your system path.

Test Gravity

See the MATLAB examples distributed with Gravity for a more complete test, but a simple test to ensure the setup is complete would be to attempt to create a Gravity Node within MATLAB:

>> path(path, '<GRAVITY_HOME>/include/MATLAB')   % replace <GRAVITY_HOME> to point to your Gravity installation
>> gravityNode = GravityNode('TEST');

To register and publish a message (after creating a GravityNode):

>> gravityNode.registerDataProduct('DPID', com.aphysci.gravity.swig.GravityTransportType.TCP);
>> gdp = GravityDataProduct('DPID');
>> gravityNode.publish(gdp);

To subscribe to a message (after creating a GravityNode):

>> sub = gravityNode.subscribe('DPID');
>> receivedDataProductList = sub.getAllDataProducts();

Note that the publish/subscribe parts of the above testing require a running ServiceDirectory. If this all executes successfully you are ready to run the more complete tests.

Uh oh!

GravitySetup

GravitySetup

Note on current and future build plans

Linux

Linux cmake instructions

Windows 64 bit build

Building Gravity

Note on using Python 3.8+ on Windows with Gravity

Using Gravity in MATLAB

Linux

MATLAB's Java version

Configure MATLAB's static java class path to include Gravity

Configure MATLAB's Java classpath

Configure MATLAB's library path

For some versions of MATLAB, it may be necessary to configure MATLAB to use the system gcc libs

Set system library path to include Gravity

Windows

Configure MATLAB's static java class path to include Gravity

Configure MATLAB's library path

Configure system path

Test Gravity

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Clone this wiki locally