ADD: Cfradial2 Reader

[syedhamidali]

• Closes can xradar read Cfradial2 netcdf files? #287, Implement open_cfradial2_datatree #93
• Tests added
• Changes are documented in history.md

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.93%. Comparing base (3f6ebc0) to head (3bf46ec).
⚠️ Report is 31 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #288   +/-   ##
=======================================
  Coverage   93.92%   93.93%           
=======================================
  Files          27       28    +1     
  Lines        5533     5539    +6     
=======================================
+ Hits         5197     5203    +6     
  Misses        336      336           

Flag	Coverage Δ
notebooktests	79.94% <83.33%> (+<0.01%)	⬆️
unittests	93.51% <100.00%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[kmuehlbauer]

@syedhamidali Just some minor suggestions, but looks good to me. We can enhance the compatibility layer later on, when users request it. Thanks!

[kmuehlbauer]

@syedhamidali Maybe my suggestion was a bit brief, but this fixture is already defined in conftest.py as well as the tmp_file fixture. You should be able to just use them here and can remove the re-definitions here.

[kmuehlbauer]

I'm not really sure, but you might have to add kwarg engine="netcdf4" here, since the tmp_file doesn't have an extension.

[aladinor]

Hi everyone,

Thanks for putting all of this together. I was looking at the cfradial2 data, and I can see it has a hierarchical structure with a root dataset as well as the sweep nodes. However, is this file fully FM301 compliant? I notice something like the root contains some attributes, but does it contain all the required attributes under the FM301 data model?

print(dtree['/'].ds)

<xarray.Dataset> Size: 40B
Dimensions:           (nsweep: 1)
Dimensions without coordinates: nsweep
Data variables:
    sweep_group_name  (nsweep) <U10 40B ...
Attributes:
    Conventions:  CF-1.7, Cf/Radial
    version:      2.0
    title:        Radar CfRadial2 Dataset
    history:      Updated CfRadial2 on 2025-05-01T00:46:09.040110Z
    RadarName:    denton.tx
    Latitude:     33.25355
    Longitude:    -97.15204
    Height:       224.0

On the other hand, I realized the dataset contains sweeps numbered as sweep_xxxx, is this the "correct" way to number the sweep under the WMO data model? Finally, I also realized this dataset contains coordinates and dimensions that I am not sure are cfradrial-FM301 compliant.

print(dtree['/sweep_0000']. ds 

<xarray.Dataset> Size: 27MB
Dimensions:                       (ntime: 723, nrange: 1003)
Dimensions without coordinates: ntime, nrange
Data variables: (12/19)
    time_us                       (ntime) datetime64[ns] 6kB ...
    range_m                       (nrange) float64 8kB ...
    azimuth_deg                   (ntime) float64 6kB ...
    elevation_deg                 (ntime) float64 6kB ...
    range_gate_width_mm           (ntime) float64 6kB ...
    start_range_mm                (ntime) float64 6kB ...
    ...                            ...
    specific_differential_phase   (ntime, nrange) float32 3MB ...
    radial_velocity_mps           (ntime, nrange) float32 3MB ...
    signal_to_noise_ratio_db      (ntime, nrange) float32 3MB ...
    spectral_width_mps            (ntime, nrange) float32 3MB ...
    filtering                     (ntime, nrange) int8 725kB ...
    rain_rate_mmhr                (ntime, nrange) float32 3MB ...
Attributes:
    NumGates:          1003
    ScanType:          2
    AntennaGain:       33.0
    AntennaBeamwidth:  2.7
    PosVelocity:       15.0

Please let me know your thoughts

[kmuehlbauer]

@aladinor See https://library.wmo.int/idviewer/35625/1042 for a description of attributes.

The thing is FM301 differs slightly from CfRadial2. How should we deal with this?

[aladinor]

Indeed, multiple aspects make this dataset quite different from the data standard. However, data generation/storage is something each institution decides. In this case, I had to do a data ingestion pipeline to put this dataset in a close version of standard-compliant format.

<xarray.DataTree>
Group: /
│   Dimensions:              ()
│   Data variables:
│       latitude             float64 8B 33.25
│       longitude            float64 8B -97.15
│       altitude             float64 8B 224.0
│       sweep_group_name     <U7 28B 'sweep_0'
│       time_coverage_start  |S32 32B b'2025-05-01T00:46:09Z'
│       time_coverage_end    |S32 32B b'2025-05-01T00:58:11Z'
│       sweep_fixed_angle    float64 8B 2.0
│   Attributes:
│       Conventions:  CF-1.7, Cf/Radial
│       version:      2.0
│       title:        Radar CfRadial2 Dataset
│       history:      Updated CfRadial2 on 2025-05-01T00:46:09.040110Z
│       scan_name:    2
├── Group: /sweep_0
│       Dimensions:                       (azimuth: 723, range: 1003)
│       Coordinates:
│         * azimuth                       (azimuth) float64 6kB 0.37 0.85 ... 359.8
│           elevation                     (azimuth) float64 6kB 2.0 2.0 2.0 ... 2.0 2.0
│           latitude                      float64 8B 33.25
│           longitude                     float64 8B -97.15
│           altitude                      float64 8B 224.0
│           crs_wkt                       int64 8B 0
│           x                             (azimuth, range) float64 6MB 0.0 ... -2.796
│           y                             (azimuth, range) float64 6MB 0.0 ... 1.001e+03
│           z                             (azimuth, range) float64 6MB 224.0 ... 259.0
│       Dimensions without coordinates: range
│       Data variables: (12/17)
│           time_us                       (azimuth) datetime64[ns] 6kB ...
│           range_m                       (range) float64 8kB ...
│           range_gate_width_mm           (azimuth) float64 6kB ...
│           start_range_mm                (azimuth) float64 6kB ...
│           carrier_frequency_hz          (azimuth) float64 6kB ...
│           pulse_length_sec              (azimuth) float64 6kB ...
│           ...                            ...
│           specific_differential_phase   (azimuth, range) float32 3MB ...
│           radial_velocity_mps           (azimuth, range) float32 3MB ...
│           signal_to_noise_ratio_db      (azimuth, range) float32 3MB ...
│           spectral_width_mps            (azimuth, range) float32 3MB ...
│           filtering                     (azimuth, range) int8 725kB ...
│           rain_rate_mmhr                (azimuth, range) float32 3MB ...
├── Group: /radar_parameters
├── Group: /georeferencing_correction
└── Group: /radar_calibration

Therefore, I think we can read "cfradial2" hierarchical files, but we can not guarantee consistency as in the other backends. Any ideas/suggestions on how to deal with this?

[kmuehlbauer]

Therefore, I think we can read "cfradial2" hierarchical files, but we can not guarantee consistency as in the other backends. Any ideas/suggestions on how to deal with this?

I'd implement a compatibility layer, a function which takes the CfRadial2 DataTree and transforms it to the wanted FM301 compatible DataTree. This can happen in the open_cfradial2_datatree function.

[aladinor]

Thanks @kmuehlbauer for mentioning this. In this draft, I created a sort of wrapper, but I think this is particular to any institution and their way to create their cfradial2 data. @syedhamidali could check it out, and perhaps we can get a nice solution for this.

[aladinor]

I feel like we can merge this, but putting a warning. Something like "cfradial2 format might not be totally FM301 compliant". This will allow xradar to read hierarchical data. Let me know your thoughts.

[kmuehlbauer]

@syedhamidali Sorry conversation has stalled. Would you want to continue here?

@aladinor Could you review this again, please?

[syedhamidali]

Hey @kmuehlbauer, yeah, we can continue here. I need to review what we can include in the compatibility layer. I believe @aladinor has been working on it, if he can walk me through the current state of the FM301 model, what’s needed, and what we’re expecting, I can get started on it again soon.

[aladinor]

Thanks for putting this together @syedhamidali! Having a CfRadial2 reader is important for xradar. Here are some observations after reviewing the code, tests, and discussion:

Design

The wrapper approach (open_datatree → open_cfradial2_datatree) follows KISS and gives API consistency with the other backends. That said, the core concern raised by @kmuehlbauer and myself in the earlier discussion is still open: CfRadial2 files from different institutions have widely varying compliance with FM301/CfRadial2 spec.

Without any validation or normalization, users will get inconsistent results:

• Dimension names vary (ntime/nrange vs azimuth/range)
• Variable names vary (azimuth_deg vs azimuth)
• Sweep naming varies (sweep_0000 vs sweep_0)
• Required root attributes may be missing

I think we should either:

• (a) Add a minimal compatibility layer (as @kmuehlbauer suggested), or
• (b) At minimum, add a clear warning in the docstring and/or emit a UserWarning that no FM301 validation is performed

Code

1. Missing type hints on the function signature:

   # Current
   def open_cfradial2_datatree(filename, **kwargs):
   
   # Suggested
   def open_cfradial2_datatree(filename: str, **kwargs) -> DataTree:

2. zarr added to requirements.txt — the implementation only uses xarray.open_datatree for NetCDF. Why is zarr a hard dependency here? If it's needed for a specific use case it should be optional.

Tests

1. Fixture re-definitions — cfradial1_file and temp_file are re-defined in the test file but already exist in conftest.py. @kmuehlbauer flagged this earlier. They should be removed and the conftest fixtures used instead.

2. Only 1 test (round-trip) — writing CfRadial1 then reading as CfRadial2. This verifies basic functionality but is missing:

   • Test with kwargs forwarding (e.g., engine parameter)
   • Error handling (invalid file path)
   • Verification of actual data values, not just structure (assert_isomorphic checks tree shape but not content)

3. Round-trip coupling — the test assumes dtree.to_netcdf() produces valid CfRadial2. If the export ever breaks, this test fails for the wrong reason. An actual CfRadial2 test file would be more robust.

Summary

The implementation is clean and minimal, which is good. The main blockers are:

• Remove fixture re-definitions (use conftest.py)
• Justify or remove zarr from requirements.txt
• Address the FM301 compliance question (compatibility layer or explicit warning)
• Strengthen tests

[kmuehlbauer]

@syedhamidali Can we close this now, as #349 is merged?