Making this project Cloud compatible and EOSC infrastructure runs

[guillaumeeb]

Hi everyone.

cc @tinaok @sebastian-luna-valero.

So I'm trying to revive this project, and to make it Cloud (Dask-gateway for the time being) compatible. I wanted to make this work apparent through this PR to get some early feedback if people have some.

In particular, I have a high level question, maybe to @rabernat: is this the good place to start a benchmarking for Cloud infrastructure performances using Pangeo? Should I look somewhere else ?

And more related to this project, I am under the impression that there have been evolutions across time that have not been reflected in Yaml configuration files, am I correct @andersy005 or @kmpaul if you ever remember? For example, trying to use cheyenne.yaml throws this:

(notebook) jovyan@jupyter-geynardbontemps:~/benchmarking$ ./pangeobench run benchmark-configs/cheyenne.yaml 
Reading configuration YAML config file
Traceback (most recent call last):
  File "/home/jovyan/benchmarking/./pangeobench", line 101, in <module>
    main()
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/jovyan/benchmarking/./pangeobench", line 44, in run
    runner.run()
  File "/home/jovyan/benchmarking/benchmarks/utils.py", line 125, in run
    operation_choice = self.params['operation_choice']
KeyError: 'operation_choice'
(notebook) jovyan@jupyter-geynardbontemps:~/benchmarking$ 

Question is: should we keep and update those files, or just throw them away?

Any other advice welcome!

[guillaumeeb]

The original issue for Performance test is there: pangeo-data/pangeo-eosc#16.

[rabernat]

This repository is completely abandoned afaik. 🙃

I have come to believe that benchmarking is a very difficult exercise to do correctly.

My latest, best effort is here: https://gallery.pangeo.io/repos/earthcube2020/ec20_abernathey_etal/cloud_storage.html

[guillaumeeb]

I have come to believe that benchmarking is a very difficult exercise to do correctly.

@rabernat Certainly!!

Thanks for the link, I'll try to take inspiration from it if I pursue here.