Extra libraries to Docker

[pstaabp]

I'm switching my development environment over to Docker (I used to have native macOs packages instead) and it seems that the basic docker install with ubuntu does not have a number of basic packages. So far I have included the following:

• a text editor (vim and nano)
• perlbrew (since I doing some testing on various versions)
• patch (needed for perlbrew)

Wondering if others have tools that they install that we should to get a more reasonable development environment. I'm happy to put together a PR to update this.

[pstaabp]

This is another related topic. When using docker, and ssh-ing into the docker environment, it doesn't appear that webwork2 or pg are git repositories. Looking at the commands to create them, they should be. Is this the case?

[taniwallach]

I'm switching my development environment over to Docker (I used to have native macOs packages instead) and it seems that the basic docker install with ubuntu does not have a number of basic packages. So far I have included the following:

• a text editor (vim and nano)
• perlbrew (since I doing some testing on various versions)
• patch (needed for perlbrew)

Wondering if others have tools that they install that we should to get a more reasonable development environment. I'm happy to put together a PR to update this.

There was a discussion in the past about having "tools" in the Docker container.

• @xcompass was opposed to having tools which are not needed in production included in the standard container image. See:
  • TO BE REPLACE: Discussions about Docker + originally hard copy fixes / extensions for WW 2.15 branch + move Docker base image to Ubuntu 18.04 LTS #972 (comment)
  • TO BE REPLACE: Discussions about Docker + originally hard copy fixes / extensions for WW 2.15 branch + move Docker base image to Ubuntu 18.04 LTS #972 (comment)

I include them at container startup using the a ADD_APT_PACKAGES setting in docker-compose.yml but this is quite inconvenient, as (a) it slows down the startup, especially when cycling up and down during testing/debugging, (b) it makes it impossible to start up the container when there is no internet connection without editing docker-compose.yml to avoid any on startup installs. I think in the end, I may have hacked docker-entrypoint.sh for a while to bypass the install block.

Given how large the image is (ex. the need for LaTeX), I strongly feel that we should reconsider including basic tools the developers want to use, at least for now, as it will make things much more pleasant and convenient for may of us. (My main includes are vim and telnet.)

@mgage @heiderich @nmoller @xcompass - Comments?

[xcompass]

Another way to solve this problem is to build a "workspace" image that includes all user tools and mount the webwork source. When doing dev, developer can login to workspace container to edit files. When in prod, simply don't run that container.

…

On Sat, Jan 25, 2020, 10:49 AM Nathan Wallach ***@***.***> wrote: I'm switching my development environment over to Docker (I used to have native macOs packages instead) and it seems that the basic docker install with ubuntu does not have a number of basic packages. So far I have included the following: - a text editor (vim and nano) - perlbrew (since I doing some testing on various versions) - patch (needed for perlbrew) Wondering if others have tools that they install that we should to get a more reasonable development environment. I'm happy to put together a PR to update this. There was a discussion in the past about having "tools" in the Docker container. - @xcompass <https://github.com/xcompass> was opposed to having tools which are not needed in production included in the standard container image. See: - #972 (comment) <#972 (comment)> - #972 (comment) <#972 (comment)> I include them at container startup using the a ADD_APT_PACKAGES setting in docker-compose.yml but this is quite inconvenient, as (a) it slows down the startup, especially when cycling up and down during testing/debugging, (b) it makes it impossible to start up the container when there is no internet connection without editing docker-compose.yml to avoid any on startup installs. I think in the end, I may have hacked docker-entrypoint.sh for a while to bypass the install block. Given how large the image is (ex. the need for LaTeX), I strongly feel that we should reconsider including basic tools the developers want to use, at least for now, as it will make things much more pleasant and convenient for may of us. (My main includes are vim and telnet.) @mgage <https://github.com/mgage> @heiderich <https://github.com/heiderich> @nmoller <https://github.com/nmoller> @xcompass <https://github.com/xcompass> - Comments? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1070?email_source=notifications&email_token=AADRVDDIATCDWPV5CI3QMK3Q7SCVJA5CNFSM4KLHOLGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJ5CZXQ#issuecomment-578432222>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADRVDDUXNLAYVKKXAWXKXTQ7SCVJANCNFSM4KLHOLGA> .

[taniwallach]

This is another related topic. When using docker, and ssh-ing into the docker environment, it doesn't appear that webwork2 or pg are git repositories. Looking at the commands to create them, they should be. Is this the case?

Actually, this is intentional. The running container is not meant to be a place from which to do development on the core system via copies of webwork2 and pg included in the container image, so there is no have the git repositories of webwork2 and pg included. Git is used to install the desired branch. Instead, developers will mount webwork2 and/or pg (or just specific files) on the container via docker-compose.yml. In fact, my production "containers" also customize the basic container image by mounting single files which have local customization.

On the other hand, the OPL is included by installing it on a persistent storage volume, so we wanted to allow using Git to update the OPL rather than requiring a full download for every update.

[taniwallach]

Another way to solve this problem is to build a "workspace" image that includes all user tools and mount the webwork source. When doing dev, developer can login to workspace container to edit files. When in prod, simply don't run that container.

@xcompass - I'm still not all that proficient in Docker. At present, I don't have time to read up myself and figure this out. Could you share some advice on how can we can create a "workspace" image which is based on the standard container image and then swap between using the production image and container image on a developers machine with minimal effort? Right now, we have a single Dockerfile in webwork2 which created the "production image", would a new Dockerfile need to be in a different subdirectory, or can it be managed via command line switches? Can the "workspace image" be done as an "overlay" to the production image?

[pstaabp]

I'm getting up to speed on docker, but I think this is possible. Here's some thoughts:

It seems according to the docker compose documentation that if we have the regular docker production file in docker-compose.yml we can create a second file, say called docker-compose-dev.yml and then to produce the development environment, then

docker-compose -f docker-compose.yml -f docker-compose-dev.yml up -d

or alternatively use the second file for production, like in this documentation, although it seems like the former makes more sense in that extra packages are needed for development.

Also, different environmental variables can be set in each compose.yml file which should also be helpful.

Here is another helpful piece of documentation on this.

Thoughts?

[drgrice1]

There is a mechanism to do this. So closing this issue.