Skip to content

Feat/fdb adapter#70

Closed
mathleur wants to merge 68 commits intofeature/rustfrom
feat/fdb_adapter
Closed

Feat/fdb adapter#70
mathleur wants to merge 68 commits intofeature/rustfrom
feat/fdb_adapter

Conversation

@mathleur
Copy link
Copy Markdown
Member

Description

Contributor Declaration

By opening this pull request, I affirm the following:

  • All authors agree to the Contributor License Agreement.
  • The code follows the project's coding standards.
  • I have performed self-review and added comments where needed.
  • I have added or updated tests to verify that my changes are effective and functional.
  • I have run all existing tests and confirmed they pass.

Base automatically changed from feat/cads_rust_adaptor to feature/rust February 24, 2026 15:29
@mathleur mathleur requested a review from Copilot March 17, 2026 09:02
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds new “meteo” adapters for building/serializing Qube structures from external list/constraint formats, while also refactoring core merge/union behavior and adding a new arena-style JSON serialization format.

Changes:

  • Add FDB and MARS list adapters plus a JSON “constraints” export adapter in qubed-meteo.
  • Refactor core merge API (unionappend / append_many) and add utilities for copying/leaf-path extraction + QubeDatacube conversion.
  • Add arena-style JSON serialization/deserialization for Qube, and reorganize float coordinates into their own module.

Reviewed changes

Copilot reviewed 18 out of 19 changed files in this pull request and generated 18 comments.

Show a summary per file
File Description
qubed/tests/test_union.rs Updates union tests to use append and adds a second test case.
qubed/src/serde/json.rs Adds arena-style JSON (de)serialization + a roundtrip test.
qubed/src/qube.rs Adds helper APIs (is_empty, node_dim, child dedup in create_child, path/copy helpers) and a new test.
qubed/src/merge.rs Renames/rewrites union into append and adds append_many.
qubed/src/datacube.rs Adds Datacube::coordinates() and Qube::to_datacubes(), wires append_datacube to append.
qubed/src/coordinates/mod.rs Moves float coordinate type into a dedicated module and updates imports.
qubed/src/coordinates/floats.rs New module for float coordinate storage/conversions.
qubed/src/compress.rs Adds a merge_subtrees helper and tweaks doc comments.
qubed/Cargo.toml Adds rayon dependency.
qubed-meteo/src/adapters/to_constraints.rs New adapter to export Qube leaf paths into DSS-like JSON constraints objects.
qubed-meteo/src/adapters/mod.rs Exposes new adapter modules.
qubed-meteo/src/adapters/mars_list.rs New parser to build a Qube from an indented MARS list format.
qubed-meteo/src/adapters/fdb.rs New parser to build a Qube from FDB-like comma-separated path strings.
qubed-meteo/src/adapters/dss_constraints.rs Changes constraints ingestion to build/merge multiple Qubes using append_many.
qubed-meteo/examples/read_from_mars_list.rs New example for timing MARS-list parsing and datacube conversion.
qubed-meteo/examples/read_from_dss_constraints.rs Updates example input + adds timing output.
qubed-meteo/Cargo.toml Adds rsfdb path dependency and a bin target configuration.
.gitignore Ignores *.list files.
.github/workflows/ci.yml Adjusts workflow triggers (adds pull_request_target labeled trigger).
Comments suppressed due to low confidence (3)

qubed/src/merge.rs:53

  • internal_set_operation returns Option<Vec<NodeIdx>> but never returns None, and callers ignore the returned value. Consider simplifying this to return () (or return a meaningful result that’s actually used) to reduce dead/unclear code paths.
    /// Performs a set operation between two groups of nodes from two Qubes.
    fn internal_set_operation(
        &mut self,
        other: &mut Qube,
        self_ids: &Vec<NodeIdx>,
        other_ids: &Vec<NodeIdx>,
    ) -> Option<Vec<NodeIdx>> {
        let mut return_vec = Vec::new();

qubed/src/coordinates/mod.rs:13

  • tiny_vec::TinyVec is imported in this module but no longer used (after moving float coordinates into floats.rs). Please remove the unused import to avoid warnings.
use std::hash::Hash;

use floats::FloatCoordinates;
use integers::IntegerCoordinates;
use strings::StringCoordinates;

use tiny_vec::TinyVec;

use crate::utils::tiny_ordered_set::TinyOrderedSet;

.github/workflows/ci.yml:29

  • The added pull_request_target trigger (on any labeled event) will run this workflow on the base repo context, but the current actions/checkout configuration checks out the default ref (base), so it won’t actually test the PR’s code. If the intent is “run CI after approval”, consider filtering on a specific label and explicitly checking out the PR merge/head ref; if you do check out PR code under pull_request_target, ensure the workflow does not expose secrets to untrusted code.
  # Trigger after public PR approved for CI
  pull_request_target:
    types: [labeled]

jobs:
  build:

    runs-on: ubuntu-latest

    steps:
    - name: Checkout code
      uses: actions/checkout@v2


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread qubed-meteo/Cargo.toml

[dependencies]
qubed = { path = "../qubed" }
rsfdb = { path = "../../rsfdb" }
Comment thread qubed/src/merge.rs
use crate::qube::Dimension;
use crate::{NodeIdx, Qube};
use std::collections::HashMap;
use std::time::Instant;
Comment thread qubed/tests/test_union.rs
Comment on lines 104 to 105
println!("{:#?}", Qube::to_ascii(&qube_a));

Comment on lines 1 to +4
pub mod dss_constraints;
pub mod fdb;
pub mod mars_list;
pub mod to_constraints;
@@ -0,0 +1,101 @@
use qubed::{Coordinates, NodeIdx, Qube};
Comment on lines +10 to +31
let datacubes: &Vec<Value> =
dss_constraints.as_array().expect("DSS constraints should be a JSON array");

let datacubes = dss_constraints.as_array().expect("DSS constraints should be a JSON array");
let order = vec![
"origin".to_string(),
"forecast_type".to_string(),
"hday".to_string(),
"day".to_string(),
"hmonth".to_string(),
"hyear".to_string(),
"year".to_string(),
"month".to_string(),
"time".to_string(),
"leadtime_hour".to_string(),
"level_type".to_string(),
"variable".to_string(),
];

for datacube in datacubes {
// Parse the first datacube and initialize the main Qube
let first_datacube = parse_datacube(&datacubes[0])?;
let mut qube = Qube::from_datacube(&first_datacube, Some(&order));

Comment thread qubed/src/qube.rs
Comment on lines +197 to +201
pub fn all_unique_dim_coords(&mut self) -> BTreeMap<String, Coordinates> {
// TODO
let mut map: BTreeMap<String, Coordinates> = BTreeMap::new();

for (_id, node) in self.nodes.iter() {
Comment thread qubed/src/compress.rs
Comment on lines +117 to +131
for (dim, source_child_ids) in source_children {
for source_child_id in source_child_ids {
let target_children = {
let target_node = self.node_ref(target_id).unwrap();
target_node.children().get(&dim).cloned().unwrap_or_default()
};

let mut merged_children = target_children.clone();
merged_children.push(source_child_id);

let target_node = self.node_mut(target_id).unwrap();
target_node.children_mut().insert(dim, merged_children.into());
}
}

Comment thread qubed/tests/test_union.rs
Comment on lines +48 to +49
println!("{:#?}", Qube::to_ascii(&qube_a));

Comment thread qubed/src/qube.rs

// collect unique coordinates per dimension
let map = qube.all_unique_dim_coords();
// only one dimension key present
@mathleur mathleur closed this Mar 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants