ece2cmor fails with updated cmor library

[klauswyser]

Running the re-cmorisation from CMIP6 to CMIP7 requires cmor=3.14. However, running ece2cmor then fails with

  File "/home/sm_wyser/cmorize/ece2cmor3/ece2cmor3/ifs2cmor.py", line 590, in define_cmor_axes
    cmor.load_table("_".join([table_root_, task.target.table]) + ".json")
                    ~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: sequence item 0: expected str instance, NoneType found
"""

When going back to cmor=3.10 I can run ece2cmor and get CMIP6 style output but then re-cmorisation fails because the old cmor library doesn't support CMIP7, e.g.

! Error: Your realization_index "r1" is invalid. 
! It must contain only characters between 0 and 9 

Cna we possibly update ece2cmor3 ?

[klauswyser]

Related to #866

[treerink]

Ok, that's a bad surprise. I fear I did not try any cmorisation, so I did not notice. I am pretty sure I read somewhere that it should have been backward compatible.

Ok let's see if we can fix it at the ece2cmor side, so we can use the latest cmor version. Hopefully not much more will pop up.

[treerink]

A solution to have them side by side running for now is to create a seperate recmorisation environment which only needs this .yml content:

name: recmorise
channels:
  - conda-forge
dependencies:
  - cmor
  - iris
  - pip
  - python
  - pip:
    - cmip7-data-request-api

[klauswyser]

The fallback solution is to have the ece2cmor3 conda env for CMIP6, and a new conda env for CMIP7. The recmorisation script doesn't need the CMIP6 version of the cmor library. It's not nice to have two different envs.

While I was typing you were just suggesting something along these lines, good.

[klauswyser]

But another thing: we need to make sure that the ece2cmor env isn't using the latest version of cmor, maybe add something like

    - cmor<=3.10

in the environment.yml file?

[treerink]

Yes limiting the cmor version is possible, probably to 3.13 I guess. But nicer if this is not necessary. But if cmor.load_table has changed... Maybe there is an option to load older tables?? Maybe with cmor_set_table, not sure.

[klauswyser]

I tested with different versions of the cmor library, ece2cmor3 works with cmor<=3.10 but not higher.

[treerink]

At hpc2020 I have been able to cmorise the nemo part of my test-all case with:

INFO:ece2cmor3.ece2cmorlib: ece2cmor3 version v2.4.0 based on the master branch
INFO:ece2cmor3.ece2cmorlib: ece2cmor_git_revision 3fb8b2cb with the long hash: 3fb8b2cb41c897cb67acec8ef1172efe28e56f26
INFO:ece2cmor3.ece2cmorlib: Python 3.14.3 | packaged by conda-forge | (main, Feb  9 2026, 21:56:02) [GC  14.3.0]
INFO:ece2cmor3.ece2cmorlib: cdo 2.5.0
INFO:ece2cmor3.ece2cmorlib: CDO_PY_VERSION: 1.6.0
INFO:ece2cmor3.ece2cmorlib: CMOR library: CMOR3.14.0
INFO:ece2cmor3.ece2cmorlib: dreqPy 01.02.00 is loaded

The ifs cmorisation is busy now.

[treerink]

For the ifs cmorisation I also got the same error as mentioned in the first post here at hpc2020.

[treerink]

Actually it looks like there is something very basic going wrong table_root_ seems to be at the initial None and then the join does crash for just making the table name. Not clear yet to me why the similar thing does work for nemo.

With a very basic example I can reproduce exactly this error message (but also for pyhton3.10.12).

[treerink]

When I tried to cmorise just Amon tas with more cores I still got the same error. And the table_root_ or tabledir is indeed often None which leads to a crash with join.

However, after seeing somewhere in the code some differences between NEMO and IFS due to the parallel handling of IFS, I tried again just Amon tas but now with one core, and that went well with a well cmorised result. And indeed correct tabledir values in between.