Unit conversion factors registered for only 15 of 61 quantity dimensions in units.txt

[jscott3201]

Problem

haystack-core/data/units.txt is a fairly complete Haystack units database — 457 unit entries across 61 -- quantity -- sections (dimensionless, currency, acceleration, angular acceleration/momentum/velocity, area, capacitance, cooling efficiency, density, electric charge/conductance/current/field strength/potential/resistance, electrical conductivity/resistivity, energy (+ apparent/reactive/by-area/by-volume), enthalpy, entropy, force, frequency, grammage, heating rate, illuminance, inductance, irradiance, length, luminance, luminous flux/intensity, magnetic field strength/flux/flux density, mass, mass flow, momentum, power (+ by-area/by-volumetric-flow/apparent/reactive), pressure, specific entropy, surface tension, temperature (+ differential), thermal conductivity, time, velocity, volume, volumetric flow, bytes). Name/symbol lookup (unit_for) covers this whole set.

But haystack-core/src/kinds/units.rs's CONVERSION_FACTORS table — the thing convert() and compatible() actually use — is a hand-written array covering only 15 quantities (temperature, length, pressure, energy, power, volume, volumetric flow, area, mass, time, electric current, electric potential, frequency, illuminance, luminous flux), roughly 90 of the 457 registered units:

static CONVERSION_FACTORS: LazyLock<HashMap<&'static str, ConversionFactor>> =
    LazyLock::new(|| {
        let entries: &[(&str, f64, f64)] = &[
            // temperature (SI base: celsius)
            ("fahrenheit", 5.0 / 9.0, -32.0 * 5.0 / 9.0),
            ...
        ];
        ...
    });

resolve() returns UnitError::UnknownUnit for any unit whose name is valid (parses via unit_for) but has no entry here — e.g. currency (USD, EUR), density (kg/m³), velocity (m/s), force (newton), entropy, capacitance, inductance, magnetic_flux, bytes, etc.

Impact

Downstream unit-bearing comparisons or conversions (e.g. normalizing energy density, flow velocity, or currency-tagged cost points) silently fail with UnitError::UnknownUnit/compatible() == false for the ~46 uncovered quantities, even though the unit names themselves are recognized elsewhere in the API (unit_for, quantity()). Verdant must special-case or app-side implement conversion for any quantity outside the 15-dimension list, and there is no documented boundary between "units known by name" and "units convertible."

Proposed direction

Either (a) generate CONVERSION_FACTORS from the same units.txt source (Project Haystack's reference units database typically ships SI conversion factors per unit) so the two tables can't drift, or (b) explicitly document/gate which quantities are convertible and have quantity()/unit_for() signal it (e.g. a has_conversion_factor(name) query) so callers can detect the gap before calling convert().

References

Project Haystack units database (units.txt reference format, quantity-sectioned).

[jscott3201]

Grounding correction before anyone picks this up: proposed direction (a) is not implementable against this repo's data file.

The issue says "Project Haystack's reference units database typically ships SI conversion factors per unit", so CONVERSION_FACTORS could be generated from units.txt and the two tables could not drift. That is true of the upstream format. It is not true of the copy in this repo.

$ grep -c ';' haystack-core/data/units.txt
0

Zero semicolons, so none of the upstream ; dim; scale; offset metadata is present. Every content line is name,symbol:

$ grep -A 3 '^-- temperature --' haystack-core/data/units.txt
-- temperature --
fahrenheit,°F
celsius,°C
kelvin,K

The 69 lines with more than two comma-separated fields are currency aliases carrying a code and a symbol, not factors:

$ awk -F',' 'NF>2' haystack-core/data/units.txt | head -3
afghani,AFN,Af
armenian_dram,AMD,Դ
aruban_guilder_florin,AWG,ƒ

And the parser matches that shape — haystack-core/src/kinds/units.rs:36 is a plain line.split(',').

So there is nothing to generate factors from. Option (a) is really two tasks: source and vendor the upstream units.txt that carries dimension/scale/offset (with a license review), rewrite parse_units for the richer grammar, and only then generate the table. That is a much larger change than the issue implies, and worth its own issue if it is wanted.

Direction for now: option (b), ratified 2026-07-25. Add a has_conversion_factor(name) query so callers can detect the boundary before calling convert() and getting a surprising UnitError::UnknownUnit for a unit whose name resolves perfectly well through unit_for(); document which quantities are convertible; and hand-extend the table where the factors are unambiguous. That does not reach all 61 quantities and is not meant to — it closes the silent surprise and leaves the full fix open.

[jscott3201]

Fixed by #42, merged to dev as 723446b.

Adversarial review (fresh Codex lane, cross-family) returned MERGE with no actionable findings; it independently searched all Rust callers of quantity, compatible, and UnitError and confirmed no workspace caller treats quantity().is_some() as convertibility. CI green on 215e854.

Closing manually: PRs target dev, so GitHub's auto-close does not fire until dev reaches main.