Investigate dockerising webots and/or running headless

[KipHamiltons]

At the moment it seems like we don't gain much performance running headless because webots renders to a non-existent screen anyway. If we can fork webots or make a branch of our fork if it still exists where we disable the stupid rendering while running headless, that would be mint.
Otherwise, dockerising it would help distribution, but alternatives to docker might be even better (docker isn't allowed on the uni's high performance computing grid, as I understand it, because the docker daemon requires root privs. @wongjoel was talking about an alternative which doesn't require root).

[wongjoel]

Singularity https://sylabs.io/guides/3.8/user-guide/ is basically "Docker for HPC" - they wrote it with running containers on HPC in mind. I've seen it used on NCI https://nci.org.au and CSIRO HPC systems, though Newcastle does not currently support it.

[KipHamiltons]

Closing this for now, since we don't plan to do this at the moment. We can reopen if that changes.

[ysims]

I think we still want to do this, just not in time for Brazil? @wongjoel

[JosephusPaye]

I thought so too.

[KipHamiltons]

We definitely can look into it, but I need more details about exactly what we want and what we're trying to achieve please

[Bidski]

If the intention is to look at an alternative for docker do we can run on the HPC grid then dont bother. No container solutions are planned for the near future as i understand it.

https://github.com/cyberbotics/webots-docker
https://hub.docker.com/r/cyberbotics/webots

[wongjoel]

Sorry I think I've caused some confusion about what we going for here. Ultimately this might even be more of a b script job than a docker job.

For optimisation, we have a bunch of individuals that need to be tested, and they can all be tested independently of each other. My current proposed architecture is to have a single "OptimisationController" that generates the individuals to evaluate, then serves those individuals to a bunch of "Evaluators". Because the Evaluators actually create the individuals by writing to disk (overriding the existing YAML file), we need the evaluators to be in different containers to not clobber each other. This is easy enough because the NUBots codebase already runs in Docker. On the webots side, we need an instance of Webots to be running for each evaluator - this instance needs to have access to the world and controller being used for optimisation, and needs to be using a matching port to what the evaluator is using.

So on the Webots/docker side (I realise some of these are just "read the docs"):

• How do we pass a custom world/controller to the docker image?
• How do we give each instance a different port? (both on the webots-server and the Nubots-client side)
• Can we automate it so we can just spin up $(nproc) containers with one command?

[Bidski]

I would also investigate using 1 webots instance with $(nproc) controllers/robots running in it. You can line them up separately so they shouldn't interfere with each other.

The NUbots side should be easy with a new b script tool or similar. You are just spinning up$(nproc) containers, each one with a different hostname and each hostname has an associated webots config file with its connection details in it