Refactoring of species and isotopologues

.clang-tidy

@@ -8,7 +8,8 @@ Checks: >
   -modernize-use-trailing-return-type,
   -readability-identifier-naming,
   performance-*,
-  portability-*
+  portability-*,
+  -llvm-qualified-auto
 
 # identifier-naming not turned on, too noisy, too many special cases
 CheckOptions:

CMakeLists.txt

@@ -488,16 +488,6 @@ else ()
   endif()
 endif ()
 
-########### Path to partition functions data ##########
-if (ARTS_PARTFUN_DATA_PATH)
-  get_filename_component (ARTS_PARTFUN_DATA_PATH "${ARTS_PARTFUN_DATA_PATH}" ABSOLUTE)
-    message(STATUS "Using manual partition functions data (unset ARTS_PARTFUN_DATA_PATH to switch to built-in options)")
-else (ARTS_PARTFUN_DATA_PATH)
-  get_filename_component (ARTS_PARTFUN_DATA_PATH "${CMAKE_CURRENT_SOURCE_DIR}/src/partfun/data" ABSOLUTE)
-  message(STATUS "Using built-in partition functions data (use -DARTS_PARTFUN_DATA_PATH=/path/to/data/ to use other partition functions)")
-endif (ARTS_PARTFUN_DATA_PATH)
-message(STATUS "The partition functions data is at ${ARTS_PARTFUN_DATA_PATH}")
-
 if (CMAKE_GENERATOR STREQUAL "Xcode")
   set (CTEST_MISC_OPTIONS -C \${CONFIGURATION})
 endif (CMAKE_GENERATOR STREQUAL "Xcode")

doc/arts/dev.species.rst

@@ -0,0 +1,152 @@
+Adding new species to ARTS
+==========================
+
+The species, isotopologues, and predefined models that ARTS support are
+defined at compile time.  This document describes the layout and the
+process of expanding the list of supported species.  The process is
+different for isotopologues and predefined models.  Both requires that
+the species already exist.
+
+To define words here:
+
+1. A species is the molecule or atom of interest, e.g., H2O, O3, CO2, etc.
+2. An isotopologue is a specific isotopic variant of a species, e.g.,
+   H2O-161, H2O-181, O3-666, etc.
+3. A predefined model is a set of parameters and equations that describe
+   the behavior of a species under certain conditions, e.g., H2O-PWR98,
+   CO2-CKDMT252, etc.
+
+All three of these are defined in individual XML files that can be read
+by a compiled version of ARTS.
+The process of adding new information generally involves adding new
+XML files to the appropriate
+directories and recompiling ARTS to make use of the new information.
+The details of the XML file format and the compilation process are
+described in the following sections.
+
+.. tip::
+   The easist way add new data of these is to
+   copy an existing files and modify the
+   relevant information.  There are a lot of
+   tests run during the CI process to ensure that
+   the basic structure of the files is correct.
+
+To add a species
+----------------
+
+Species are defined in ``arts-cat-data/species/``.
+They must be named in an index-ordered manner, e.g., ``0001.H2O.xml``,
+``0002.CO2.xml``, etc.
+Missing numbers are allowed but not recommended -
+it is used during the CI process.
+
+The file format is a :class:`~pyarts3.arts.SpeciesEnumInfo` XML file.
+The information inside the file is the identifying index of the species,
+the short name that humans are expected to use, and a long name that is used
+primarily to define the value of the species in
+the :class:`~pyarts3.arts.SpeciesEnum` class.
+The short name is used in the ARTS XML files to refer to the species.
+
+The reason these are different is that some species such
+as ``NO+`` cannot be used
+as C++/Python identifiers, making it cumbersome to write
+automatic code generation from an enumeration.
+
+To add an isotopologue
+----------------------
+
+Isotopologues are defined in ``arts-cat-data/isotopologues/``.
+They must be named by the species they contain and the isotopologue
+number that identifies the specific isotopologue,
+e.g., ``H2O-161.xml``, ``H2O-181.xml``, etc.
+
+The file format is a :class:`~pyarts3.arts.SpeciesIsotopologueInfo` XML file.
+The content of these files is the short name of the species,
+the isotopologue index,
+the mass in atomic mass units of the isotoplogue,
+the default isotopologue ratio in
+Earth's atmosphere, and the degeneracy of the molecular lines.
+
+You are not done yet, however, as you also need to add the isotopologue to
+the partition function files, and possibly to
+the HITRAN and/or JPL compatibility files.
+
+To add a partition function
+^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The partition function files are defined
+in ``arts-cat-data/partition-functions/``.
+They must also be named by the species they contain and the isotopologue
+number that identifies the specific isotopologue,
+e.g., ``H2O-161.xml``, ``H2O-181.xml``, etc.
+
+The partition function files are
+a :class:`~pyarts3.arts.PartitionFunctionsData` XML file.
+This is a pure data file that contains a tag for the format of the
+partition function data, and a simple :class:`~pyarts3.arts.Matrix`
+to hold the data.  The supported formats are defined by
+the :class:`~pyarts3.arts.PartitionFunctionsType` enumeration.
+See either type for more information on the supported
+formats and the expected data layout.
+
+To add support for HITRAN
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+This is required if you want to read a HITRAN line list containing the
+isotopologue you want to add.
+
+Supported HITRAN isotopologues are defined in ``arts-cat-data/hitran/``.
+They must be named by the isotopologue the add
+support for, e.g., ``H2O-161.xml``, ``H2O-181.xml``, etc.
+
+The file format is a :class:`~pyarts3.arts.HitranSpeciesInfo` XML file.
+The content of these files is the full name of the isotopologue,
+the index of the species in the HITRAN database, and the tag character
+of the isotopologue in the HITRAN database, and the HITRAN-defined
+isotopologue ratio.
+The latter is required to compute the Einstein A coefficients
+from the HITRAN line strengths.
+
+.. tip::
+
+   If you need quantum numbers, use the HITRAN online database
+   to append ``qns'`` and ``qns''`` at the end of the ``par-line``
+   format.
+
+To add support for JPL
+^^^^^^^^^^^^^^^^^^^^^^
+
+This is required if you want to read a JPL line list containing the
+isotopologue you want to add.
+
+Supported JPL isotopologues are defined in ``arts-cat-data/jpl/``.
+They must be named by the isotopologue the add support
+for, e.g., ``H2O-161.xml``, ``H2O-181.xml``, etc.
+
+The file format is a :class:`~pyarts3.arts.JplSpeciesInfo` XML file.
+The content of these files is the index identifier of the
+isotopologue in the JPL database,
+the full name of the isotopologue, the reference temperature of
+the line strengths in the JPL database, the JPL-defined isotopologue ratio,
+and a currently unused tag for whether the quantum number format is supported.
+
+.. admonition:: FIXME
+   :class: danger
+
+   The quantum numbers are currently completely ignored.  We would
+   be happy to add support for them but will likely require external
+   help to do so.
+
+To add a predefined model
+-------------------------
+
+The predefined models are not defined in the
+``arts-cat-data/`` directory, but is part of the
+ARTS source code tree.  They are
+defined in ``arts/src/core/spec/predef-models/``.
+The must be named by the species they contain and the model name, e.g.,
+``H2O-PWR98.xml``, ``CO2-CKDMT252.xml``, etc.
+
+The file format is also
+a :class:`~pyarts3.arts.SpeciesIsotopologueInfo` XML file,
+but only the short name of the species and the isotopologue names are used.

doc/arts/devs.rst

@@ -23,3 +23,4 @@ and how to use existing features.
    dev.gui
    dev.doc
    dev.xml
+   dev.species

python/src/pyarts3/hitran.py

@@ -9,6 +9,19 @@
 }
 
 
+def pos2char(ind):
+    """ Convert an isotoplogue index to a HITRAN char for that index """
+    if ind < 10:
+        return "'{}'".format(ind)
+    elif ind == 10:
+        return "'0'"
+    elif ind == 11:
+        return "'A'"
+    elif ind == 12:
+        return "'B'"
+    raise ValueError("Invalid isotopologue index: {}".format(ind))
+
+
 def get_latest_molparam_map(molparam_txt_file=None):
     """ Generates a version of latest_molparam_map used in hitran_species.cc
 
@@ -20,16 +33,6 @@ def get_latest_molparam_map(molparam_txt_file=None):
     3. isotopologue string: isotopologue identifier in HITRAN notation
     4. abundance: natural abundance of the isotopologue
     """
-    def pos2char(ind):
-        """ Convert an isotoplogue index to a HITRAN char for that index """
-        if ind < 10:
-            return "'{}'".format(ind)
-        elif ind == 10:
-            return "'0'"
-        elif ind == 11:
-            return "'A'"
-        elif ind == 12:
-            return "'B'"
 
     if molparam_txt_file:
         molparam_txt = open(molparam_txt_file, 'r').read().split('\n')
@@ -55,6 +58,28 @@ def pos2char(ind):
     return out
 
 
+def to_cat_data(molparam_txt_file=None):
+    """ Converts the molparam.txt file to a format suitable for catdatabase input.
+
+    The output is printed to stream with print() as the intent is to use this
+    output in ARTS directly.
+    """
+
+    import pyarts3 as pyarts
+    data = get_latest_molparam_map(molparam_txt_file)
+
+    out = []
+    for spec in data:
+        for entry in data[spec]:
+            x = pyarts.arts.HitranSpeciesInfo()
+            x.spec = pyarts.arts.SpeciesIsotope(spec + '-' + entry[2])
+            x.hitind = int(entry[0])
+            x.hitchar = pos2char(entry[1])[1]
+            x.ratio = float(entry[3])
+            out.append(x)
+    return out
+
+
 def gen_latest_molparam_map(molparam_txt_file=None):
     """ Generates a version of latest_molparam_map used in hitran_species.cc
 

src/core/artstime/CMakeLists.txt

@@ -1,7 +1,7 @@
 add_library(artstime STATIC artstime.cc)
 
 target_include_directories(artstime PUBLIC "${CMAKE_CURRENT_SOURCE_DIR}")
-target_link_libraries(artstime PUBLIC time_report xml_io FastFloat::fast_float)
+target_link_libraries(artstime PUBLIC time_report arts_enum_options xml_io FastFloat::fast_float)
 
 ####################################################################################
 

src/core/artstime/artstime.cc

@@ -9,7 +9,7 @@
 #include "artstime.h"
 
 #include <debug.h>
-#include <enumsTimeStepType.h>
+#include <enums.h>
 #include <fast_float/fast_float.h>
 
 #include <charconv>

src/core/functional/CMakeLists.txt

@@ -8,7 +8,7 @@ add_library(functional_numeric_ternary STATIC
   functional_atm.cpp
 )
 
-target_link_libraries(functional_numeric_ternary PUBLIC matpack geodesy artstime igrf)
+target_link_libraries(functional_numeric_ternary PUBLIC matpack geodesy arts_enum_options artstime igrf)
 target_include_directories(functional_numeric_ternary PUBLIC ${CMAKE_CURRENT_SOURCE_DIR})
 
 ### META-package

src/core/functional/functional_atm_pressure_field.h

@@ -1,6 +1,7 @@
 #pragma once
 
 #include <artstime.h>
+#include <enums.h>
 #include <enumsFieldComponent.h>
 #include <matpack.h>
 #include <xml.h>

src/core/geodesy/geodetic.cpp

@@ -668,7 +668,7 @@ bool terrain_occludes(Numeric obs_lat,
 }  // anonymous namespace
 
 std::vector<Vector2> visible_coordinates(Vector2 pos,
-                                         Vector2 ellipsoid,
+                                         Vector2,
                                          const GeodeticField2& hfield) {
   if (not hfield.ok()) return {};
 

src/core/lbl/CMakeLists.txt

@@ -2,6 +2,7 @@ add_library(lbl STATIC
   lbl_data.cpp
   lbl_fwd.cpp
   lbl_hitran.cpp
+  lbl_jpl.cpp
   lbl_lineshape.cpp
   lbl_lineshape_linemixing.cpp
   lbl_lineshape_model.cpp

src/core/lbl/lbl.h

@@ -3,6 +3,7 @@
 #include "lbl_data.h"
 #include "lbl_fwd.h"
 #include "lbl_hitran.h"
+#include "lbl_jpl.h"
 #include "lbl_lineshape.h"
 #include "lbl_lineshape_model.h"
 #include "lbl_lineshape_voigt_ecs.h"

src/core/lbl/lbl_data.cpp

@@ -31,6 +31,14 @@ using std::exp;
 using std::expm1;
 
 namespace lbl {
+Numeric einstein_a(
+    Numeric s, Numeric gu, Numeric e0, Numeric f0, Numeric T, Numeric Q) {
+  //! Note negative value because expm1 is used as a more accurate form of (1 - exp(x)) for exp(x) close to 1.
+
+  return -8.0 * pi * Q * s /
+         (gu * exp(-e0 / (k * T)) * expm1(-(h * f0) / (k * T)) * pow2(c / f0));
+}
+
 void band_data::sort(LineByLineVariable v) {
   using enum LineByLineVariable;
   switch (v) {
@@ -150,10 +158,7 @@ const Numeric& line_key::get_value(const AbsorptionBands& b) const {
                                       const Numeric T0) const {
   const Numeric Q0 = PartitionFunctions::Q(T0, isot);
 
-  //! Note negative value because expm1 is used as a more accurate form of (1 - exp(x)) for exp(x) close to 1.
-  return -8.0 * pi * Q0 * s /
-         (gu * exp(-e0 / (k * T0)) * expm1(-(h * f0) / (k * T0)) *
-          pow2(c / f0));
+  return einstein_a(s, gu, e0, f0, T0, Q0);
 }
 
 Numeric line::hitran_a(const Numeric hitran_s,

src/core/lbl/lbl_data.h

@@ -25,6 +25,9 @@
 #include "lbl_zeeman.h"
 
 namespace lbl {
+Numeric einstein_a(
+    Numeric s, Numeric gu, Numeric e0, Numeric f0, Numeric T, Numeric Q);
+
 struct line {
   //! Einstein A coefficient
   Numeric a{};