Add WisdomAI domain converter (both directions)#239
Conversation
Adds converters/wisdom, a one-way converter from a WisdomAI domain export JSON (format 1.0) to an Ossie semantic model YAML, following the layout of the dbt converter. Mapping: domain -> semantic model; domain knowledge and system instructions -> model-level ai_context; tables/columns/formulas -> datasets/fields; relationship graph -> relationships (cardinality folded into edge direction, compound AND-joins flattened to composite keys); per-table measures -> model-level metrics. Expressions are emitted verbatim under the dialect mapped from the table's connection (snowflake, databricks), falling back to ANSI_SQL with a warning for unsupported dialects. Information loss is reported through the same ConverterResult/ConverterIssue pattern as ossie-dbt. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds WISDOM to the OSIVendor enum, the vendor examples in the spec documents and JSON schema, and the supported-vendors table in the converter guide. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds osi-to-wisdom, the reverse of the wisdom converter: an Ossie document becomes a wisdom domain export (format 1.0) consumable by wisdom's importDomain API. The mapping inverts wisdom-to-osi so round-trips are stable in both directions: model ai_context splits back into system instructions and knowledge items, fields split into columns and formulas, relationship direction is read as many-to-one with ai_context notes restoring one-to-one/many-to-many and composite keys becoming compound AND join conditions, and metrics attach to the table their expression references. IDs are deterministic (derived from names) and connections are per-dialect placeholders remapped at import time. Verified: Ossie -> wisdom -> Ossie round-trips of two real domain exports are byte-identical. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
BigQuery connections now map to the BIGQUERY dialect added to the spec instead of falling back to ANSI_SQL with a warning, with backtick identifier quoting; both directions round-trip it. The pyproject follows the repo's UV layout (dependency-groups, uv sources for the in-repo apache-ossie). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
| from ossie_wisdom.osi_to_wisdom import OSIToWisdomConverter | ||
| from ossie_wisdom.wisdom_to_osi import WisdomToOSIConverter | ||
|
|
||
| _ISSUE_REASON: dict[ConverterIssueType, str] = { |
There was a problem hiding this comment.
The dict is manually kept in sync with ConverterIssueType. Any new type added to the enum without a corresponding entry will crash the CLI at runtime with a KeyError.
I suggest to use .get() with a fallback:
reason = _ISSUE_REASON.get(issue.issue_type, issue.issue_type.value)
There was a problem hiding this comment.
Good catch — fixed in 63f125b using .get() with the enum value as the fallback reason, so a missing entry degrades to a terse-but-correct warning instead of a KeyError.
|
|
||
|
|
||
| def _stable_id(prefix: str, value: str) -> str: | ||
| return f"{prefix}_{hashlib.md5(value.encode('utf-8')).hexdigest()}" |
There was a problem hiding this comment.
On FIPS-compliant systems, MD5 raises ValueError by default. Since this is for ID generation (not security), adding usedForSecurity=False makes the intent clear and avoids the crash:
| return f"{prefix}_{hashlib.md5(value.encode('utf-8')).hexdigest()}" | |
| return f"{prefix}_{hashlib.md5(value.encode('utf-8'), usedForSecurity=False).hexdigest()}" |
There was a problem hiding this comment.
Agreed and fixed in 63f125b. One note: the Python keyword is lowercase usedforsecurity (the camelCase usedForSecurity from the suggestion raises TypeError), so I applied it with the correct spelling plus a comment explaining it's for FIPS-enabled builds.
Use .get() with the enum value as fallback when printing converter warnings so a ConverterIssueType without a reason entry cannot crash the CLI, and pass usedforsecurity=False to hashlib.md5 so deterministic ID generation works on FIPS-enabled builds. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Supersedes #192, which was closed during the repository migration (the old repo is now
apache/ossie-temp, and its head points at a stale mirror branch). This is the same converter rebased onto the currentmain, with two adaptations to changes that landed since:BIGQUERYdialect (with backtick identifier quoting) instead of falling back toANSI_SQLwith a warning.pyproject.tomlfollows the repo's UV conventions (dependency-groups,tool.uv.sourcesfor the in-repoapache-ossie).What
Adds
converters/wisdom, a bidirectional converter between WisdomAI domain exports (format1.0, the JSON produced by wisdom'sexportDomainAPI and consumed byimportDomain) and Ossie semantic model YAML. It follows the layout and conventions of the dbt converter (apache-ossiedependency,ConverterResult/ConverterIssueloss reporting, argparse CLI). Also registersWISDOMas a well-known custom-extension vendor in the spec documents andOSIVendorenum.Import mapping (wisdom → Ossie)
semantic_model[].name/descriptionref.name/descriptionai_context(model level)domainSystemInstructions+ domain knowledge contents as a bulleted listdatasets[]location.database.schema.dbTable→source;primaryKey.columnsorisPrimaryKeyflags →primary_key)datasets[].fields[]displayName→label; DATE/DATETIME/TIMESTAMP →dimension.is_timerelationships[]from= many side); compound AND-joins flattened to composite key arrays; OR/non-equi joins dropped with a warningmetrics[]Expressions are emitted verbatim under the Ossie dialect mapped from the table's connection (
snowflake → SNOWFLAKE,databricks → DATABRICKS,bigquery → BIGQUERY); other dialects fall back toANSI_SQLverbatim with anUNSUPPORTED_DIALECTwarning.Export mapping (Ossie → wisdom)
The exact inverse, so round-trips are stable in both directions:
ai_contextsplits back intodomainSystemInstructions(leading text) + one knowledge item per-bullet.dimension.is_timemaps to aTIMESTAMPdata type (wisdom re-derives exact types from the warehouse).MANY_TO_ONEedges; theai_contextnotes written by the import direction restoreONE_TO_ONE/MANY_TO_MANY, and composite keys become compoundANDjoin conditions.unique_keys,custom_extensions, andai_contexton fields/metrics (plus synonyms/examples anywhere).Testing
OSI → wisdom → OSIyields an identicalOSIDocument), and determinism of the emitted export.validation/validate.pyfully (schema, uniqueness, reference integrity, sqlglot SQL parsing), andOSI → wisdom → OSIround-trips are byte-identical YAML for both.🤖 Generated with Claude Code