HYRAX-1964: Build images from bes base docker image (bes_core) instead of bes and libdap rpms

[hannahilea]

Images to update (all for rhel8):

• hyrax basic
• besd
• hyrax_ncwms
• ngap

Before merge:

• test el8 on SIT

Note: does NOT update the rhel9 builds to build from this bes_core base image. That work will need to happen in another branch.

[hannahilea]

...to prevent temporarily breaking el9 builds on master, we may want to consider reviewing and merging #120 into this branch before merging this branch down.

[hannahilea]

[hannahilea]

Current sizes, calculated via running the below script [1] on my EC2 instance:

• el-8:

IMAGE                                            ID             DISK USAGE   CONTENT SIZE
opendap/besd:3.21.1-1106-el8                     f8f59d31fe0b       2.08GB          640MB
opendap/besd:3.21.1-1106-el8-test-deploy         d9fe4bd6fdf7          3GB          680MB
opendap/hyrax:1.17.1-733-el8                     6771d27ee72f       2.77GB          784MB
opendap/hyrax:1.17.1-733-el8-test-deploy         23f0ea0dea3b       4.11GB         1.03GB
opendap/hyrax:ngap-1.17.1-733-el8                86bbfa0f328a       3.34GB          973MB
opendap/hyrax:ngap-1.17.1-733-el8-test-deploy    568b3e6a8357        4.4GB          1.1GB
opendap/hyrax_ncwms:1.17.1-733-el8               b8f9d36afcd9       2.87GB          828MB
opendap/hyrax_ncwms:1.17.1-733-el8-test-deploy   e3bf7330b51a       4.21GB         1.07GB

• el-9

IMAGE                                            ID             DISK USAGE   CONTENT SIZE
opendap/besd:3.21.1-1106-el9                     dc8483052bfe        946MB          283MB
opendap/besd:3.21.1-1106-el9-test-deploy         bd6d66033214       2.37GB          529MB
opendap/hyrax:1.17.1-734-el9                     a0133a832494       2.51GB          734MB
opendap/hyrax:1.17.1-734-el9-test-deploy         87270ba2d549       3.85GB          961MB
opendap/hyrax:ngap-1.17.1-734-el9                58b957a0be19       3.35GB          952MB
opendap/hyrax:ngap-1.17.1-734-el9-test-deploy    1393689b13ad        4.8GB         1.23GB
opendap/hyrax_ncwms:1.17.1-734-el9               0121f9a1f1e0       2.51GB          734MB
opendap/hyrax_ncwms:1.17.1-734-el9-test-deploy   891d58666771       3.85GB          961MB

[1] I first cleaned out all images on my machine (docker system prune -a). I ended up only doing two of the pull_images_for_tag at a time, with a clean in between, because my machine ran out of space when pulling them all down together.

#!/bin/sh
set -e

function pull_images_for_tag(){
    local prefix=$1
    local tag=$2
    local besd_tag=$3
    echo "$prefix: Pulling images with version tag '$tag'"

    docker pull opendap/hyrax:${tag}
    docker pull opendap/besd:${besd_tag}
    docker pull opendap/hyrax:ngap-${tag}
    docker pull opendap/hyrax_ncwms:${tag}
}

docker image ls

# Build 3133: https://app.travis-ci.com/github/OPENDAP/hyrax-docker/builds/277628480
pull_images_for_tag "Master-el8" "1.17.1-733-el8" "3.21.1-1106-el8"

# Build 3136: https://app.travis-ci.com/github/OPENDAP/hyrax-docker/builds/277629035
pull_images_for_tag "New-el8" "1.17.1-733-el8-test-deploy" "3.21.1-1106-el8-test-deploy"

# Build 3134: https://app.travis-ci.com/github/OPENDAP/hyrax-docker/builds/277628568
pull_images_for_tag "Master-el9" "1.17.1-734-el9" "3.21.1-1106-el9"

# Build 3137: https://app.travis-ci.com/github/OPENDAP/hyrax-docker/builds/277629062
pull_images_for_tag "New-el9" "1.17.1-734-el9-test-deploy" "3.21.1-1106-el9-test-deploy"

docker image ls

[hannahilea]

I was having a hard time interpreting the table, so I turned it into a couple charts.

[hannahilea]

Testing proposal from @ndp-opendap via Slack:

w.r.t. testing:

• opendap/hyrax - hyrax_regression_tests

• ✅ Passed in CI; passed in Bamboo build of test-deploy branch

• opendap/besd - We could use (the possibly antiquated) docker compose to mate it to opendap/olfs while mounting the test data on /usr/share/hyrax and then run the hyrax_regression_tests, like locally. Or, get opendap/besd, run it exposing port 10022 and mounting the test data on /usr/share/hyrax. Then  just run the OLFS on a local Tomcat and then run the hyrax_regression_tests??

• In progress!
• I did do a docker run -it opendap/besd:snapshot-el8-test-deploy /bin/bash (and also -el9) to see that the besdaemon successfully launched, which it did

• opendap/hyrax_ncwms - This is identical to opendap/hyrax but has the NcWMS service running as well. The hyrax_regression_tests don't do a thing with the NcWMS part, so there are no tests for this. Given that I am debating removing all of this NcWMS stuff I think you should just skip it for now.

• Skipping, as recommended

• opendap/hyrax:ngap- - hyrax_regression_tests, deploy to SIT, Hyrax-500 tests?

• ✅ Deployed test-deploy branch to SIT, ran hyrax500 tests successfully

[hannahilea]

Copying over relevant questions from @ndp-opendap in #123:

This is making me realize that without the RPMs the checkpoints are more fluid...

Do you have a sense of how the build version and all this will be affected?

Like with this docker build architecture are we able to get the Hyrax version response from a running server and easily find the commit of each component that was used to craft the docker image??

[hannahilea]

This is making me realize that without the RPMs the checkpoints are more fluid...
Do you have a sense of how the build version and all this will be affected?

I don't think that it should be affected at all? I've tagged the bes_core image following the same versioning conventions as we used in naming the RPMs. But maybe I'm missing something important here?

Like with this docker build architecture are we able to get the Hyrax version response from a running server and easily find the commit of each component that was used to craft the docker image??

Great question---I didn't know that functionality even existed!! I will test for it.

• H: test request on new image

[hannahilea]

On my EC2 instance, did

 docker run -v $PWD/olfs:/etc/olfs -v /home/ubuntu/web.xml:/usr/share/tomcat/webapps/opendap/WEB-INF/web.xml -p 8080:8080 -d --name hyrax opendap/hyrax:snapshot-el8-test-deploy

which pulled down the test image built by Travis.

Did

curl -v -L -n -c cookies -b cookies -o foo "http://localhost:8080/opendap/hyrax/version"

which resulted in

<?xml version="1.0" encoding="UTF-8"?>
<HyraxCombinedVersion>
  <Hyrax version="1.17.1-733" />
  <OLFS version="1.18.15-733" />
  <BesGroup prefix="/">
    <BES xmlns="http://xml.opendap.org/ns/bes/1.0#" prefix="/" name="/">
      <Administrator>support@opendap.org</Administrator>
      <library name="bes" version="3.21.1-1106" />
      <module name="asciival" version="4.2.6" />
      <module name="csv_handler" version="1.2.9" />
      <library name="libdap" version="3.21.1-390" />
      <serviceVersion name="dap">
        <version>2.0</version>
        <version>3.0</version>
        <version>3.2</version>
      </serviceVersion>
      <module name="dmrpp_module" version="1.1.8" />
      <module name="freeform_handler" version="3.10.9" />
      <module name="fileout_covjson" version="1.1.0" />
      <module name="fileout_json" version="1.1.9" />
      <module name="fileout_netcdf" version="1.5.6" />
      <module name="functions" version="1.1.2" />
      <module name="gateway" version="0.2.0" />
      <module name="hdf4_handler" version="1.16.9" />
      <module name="hdf5_handler" version="1.16.9" />
      <module name="netcdf_handler" version="3.12.8" />
      <module name="ncml_moddule" version="1.5.8" />
      <module name="dapreader_module" version="0.0.1" />
      <module name="s3_reader" version="0.9.2" />
      <module name="bes" version="3.21.1" />
      <module name="usage" version="4.3.7" />
      <module name="w10n_handler" version="1.1.9" />
      <module name="xml_data_handler" version="1.2.5" />
    </BES>
  </BesGroup>
</HyraxCombinedVersion>

This looks like what we'd expect, yes? It looks like the versions intentionally don't contain build numbers, which I'd maybe want them to contain, but that's not a change in behavior compared to the deployed version.... I missed that we have bes in there as a library (with build number) AND module (without). Great!

[hannahilea]

...last test to do here is the besd test @ndp-opendap proposed above, which I'm stuck on. I also don't know how important it is, given that running the image without the olfs did start the daemon. If it's important, I will need help running it!

[hannahilea]

...and el9 version:

<?xml version="1.0" encoding="UTF-8"?>
<HyraxCombinedVersion>
  <Hyrax version="1.17.1-734" />
  <OLFS version="1.18.15-734" />
  <BesGroup prefix="/">
    <BES xmlns="http://xml.opendap.org/ns/bes/1.0#" prefix="/" name="/">
      <Administrator>support@opendap.org</Administrator>
      <library name="bes" version="3.21.1-1106" />
      <module name="asciival" version="4.2.6" />
      <module name="csv_handler" version="1.2.9" />
      <library name="libdap" version="3.21.1-390" />
      <serviceVersion name="dap">
        <version>2.0</version>
        <version>3.0</version>
        <version>3.2</version>
      </serviceVersion>
      <module name="dmrpp_module" version="1.1.8" />
      <module name="freeform_handler" version="3.10.9" />
      <module name="fileout_covjson" version="1.1.0" />
      <module name="fileout_json" version="1.1.9" />
      <module name="fileout_netcdf" version="1.5.6" />
      <module name="functions" version="1.1.2" />
      <module name="gateway" version="0.2.0" />
      <module name="hdf4_handler" version="1.16.9" />
      <module name="hdf5_handler" version="1.16.9" />
      <module name="netcdf_handler" version="3.12.8" />
      <module name="ncml_moddule" version="1.5.8" />
      <module name="dapreader_module" version="0.0.1" />
      <module name="s3_reader" version="0.9.2" />
      <module name="bes" version="3.21.1" />
      <module name="usage" version="4.3.7" />
      <module name="w10n_handler" version="1.1.9" />
      <module name="xml_data_handler" version="1.2.5" />
    </BES>
  </BesGroup>
</HyraxCombinedVersion>

[hannahilea]

Ran hyrax_regression_tests on besd and hyrax_ncwms images as described above (w/ndp); success!

[hannahilea]

Okay, last step before merging: ndp proposed reaching out to a downstream team to make sure that the besd image size change won't be a problem for them. Doing that now.